System and Method for Using Voice Over a Telephone to Access, Process, and Carry Out Transactions Over the Internet

ABSTRACT

A method for executing a transaction related to an item or a service using a telephone includes providing information identifying the item or the service, providing a query as to a transaction to be performed in which the transaction is related to the identified item or service, and sending to a server system a request to execute the transaction related to the identified item or service in response to a user answer. The transaction is executed without the user performing a single action on a computer interface.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.09/532,509, filed on Mar. 21, 2000 which is incorporated by referenceherein.

BACKGROUND OF THE INVENTION

The present invention relates generally to the Internet and electroniccommerce, or “e-commerce”. More particularly, the present inventionrelates to a system and method for using voice over a telephone toaccess, process, and carry out transactions over the Internet.

The Internet has developed into a medium by which a person using acomputer connected to the Internet can access voluminous amounts ofinformation. The ability to access information via the Internet can beprovided in a variety of different ways. Sometimes information isprovided by Internet search engines, which typically search the Internetfor key words or phrases and then provide a list of web sites whichinclude the search words or phrases in the web page, such as, its textor embedded identifiers (e.g., metatags). Information is also accessiblevia the Internet by individual web sites. Individual web sites provide awide variety of information and services which are both time-criticaland not time dependent.

The Internet is especially conducive to conducting electronic commerce.Many Internet servers have been developed through which vendors canadvertise and sell their products or services. Such products or servicesmay include items (e.g., music) that are delivered electronically to thepurchaser over the Internet and items (e.g., books) that are deliveredthrough conventional distribution channels (e.g., a common carrier). Theservices can include providing information (e.g., weather, traffic,movies, cost comparisons) that is available over the Internet andtransactions (e.g., stock trading, restaurant reservations) that arecarried out over the Internet.

Unfortunately, while the Internet provides users with the potential toaccess a tremendous amount of information, finding useful Internet-basedinformation is often time-consuming and cumbersome. Further, it isdifficult to find and compare the same information available at multipleindividual web sites because the same information can be organized inmany different ways, described in many different forms, and changed atmany different times. Added to these inherent difficulties with theInternet is the simple fact that a person cannot access the informationavailable on the Internet without having a computer or other suchelectronic device which is connected to the Internet via an InternetService Provider (ISP). Furthermore, to effectively find desiredInternet-based information, a person must learn how to locateinformation via the Internet. As such, persons without computers, peoplewithout connections to ISPs, people without appropriate software, andpeople without experience or training on use of the Internet are limitedfrom access to Internet-based information. These factors contribute toreasons why industry experts estimate that by the end of 1999, only 30%of the United States population has ever accessed the Internet, or“surfed the web.” (Statistics from Forrester Research, October 1999).

Hence, it is desirable to provide a system and method by which peoplecan access Internet-based information without directly using a computer,having a personal ISP connection, or gaining experience or training onuse of the Internet. In addition, it is desirable to provide a systemand method which allows people to obtain Internet-based informationusing convenient and readily available means, such as, by way of voiceover a public telephone. Further, it is desirable to provide a systemand method which allows for using voice over a telephone to access,process, and carry out transactions over the Internet. Even further,such transactions should be possible with any user interface platform.

Many challenges have heretofore made such a system and methodimpossible. For example, people using such a system and method wouldwant to have the information quickly or, at least, within some tolerableamount of time. Such speed is difficult. Even with conventionally highspeed computers and fast communication connections, the delay requiredto access the Internet has made many people call it the “world widewait” instead of the “world wide web.” Another challenge to such asystem and method is the recognition of voice communications.Conventional voice recognition technology is slow and inaccurate.Convenient and meaningful access to Internet-based information by voicewould require simple, quick, and accurate voice recognition.Nevertheless, known processors and memory devices do not allow quickaccess to the large vocabularies and processing speeds which would benecessary for voice recognition as done in human-to-human interaction.

Yet another challenge to such a system and method is how to provide freeaccess to Internet-based information while financially supporting theservice. Conventional advertising on the Internet requires the abilityto see advertising information, such as “banners”, and make some manualselection, such as “clicking” the banner, to get more information on theadvertised product or service.

Therefore, in addition to the above-mentioned capabilities, it isdesirable to provide a system and method by which people can gain quickand accurate voice access to Internet-based information free of charge.It is further desirable to provide a system and method by which theinformation and processing necessary for Internet-based transactions isprovided to users of a variety of platforms, including speech andwireless access protocol (WAP).

BRIEF SUMMARY OF THE INVENTION

One aspect of an embodiment of the invention is a method for executing atransaction related to an item or a service using a telephone. Themethod includes providing information identifying the item or theservice, providing a query as to a transaction to be performed in whichthe transaction is related to the identified item or service, andsending to a server system a request to execute the transaction relatedto the identified item or service in response to a user answer. Thetransaction is executed without the user performing a single action on acomputer interface.

Briefly, another aspect of an embodiment of the invention is avoice-controlled transaction service which processes transactions overthe Internet. The service includes a user interface and at least onedatabase.

The user interface coordinates voice communications with a user in whichthe voice communications include item or service information andtransactions associated with the item or service. The database iscoupled to the user interface and stores item and service information.The transactions are executed without the user pressing a button,clicking a mouse, or any other manual input to a computing device.

Briefly, another aspect of an embodiment of the invention is a servicefor providing access to Internet-based information and execution ofInternet-based transactions where the user communicates with the serviceusing voice over a telephone. The service includes means for providinginformation identifying an item or a service and providing a query as toa transaction to be performed which is related to the identified item orservice; and means for sending to a server system a request to executethe transaction related to the identified item or service in response toa user answer to the provided question.

Briefly, another aspect of an embodiment of the invention is a computerprogram product comprising computer readable program code for executionof a transaction related to an item or a service using a communicationdevice. The program code in the computer program product includes firstcomputer readable program code for providing information identifying theitem or the service, second computer readable program code for providinga query as to a transaction to be performed related to the identifieditem or service, and third computer readable program code for sending toa server system a request to execute the transaction related to theidentified item or service in response to the user response to thequery. The transaction is executed without the user performing a manualaction.

Other principle features and advantages of the present invention willbecome apparent to those skilled in the art upon review of the followingdrawings, the detailed description, and the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is illustrated by way of example and not limitation in thefigures of the accompanying drawings, in which like references indicatesimilar elements and in which:

FIG. 1 is a general diagrammatical representation of a voice portalconnected to the Internet;

FIG. 2 is a general functional block diagram of an exemplary functionalembodiment of the voice portal of FIG. 1;

FIG. 3 is a more detailed block diagram of an exemplary physicalembodiment of the voice portal of FIG. 1;

FIG. 4 is a diagrammatical representation of an exemplary data structuremodel used by the voice portal of FIG. 1;

FIG. 5 is a diagrammatical representation of the exemplary datastructure model of FIG. 4 for user related information;

FIG. 6 is a diagrammatical representation of the exemplary datastructure model of FIG. 4 for advertising related information;

FIG. 7 is a flow diagram illustrating an exemplary creation process ofthe exemplary data structure model of FIG. 4;

FIG. 8 is a diagrammatical representation of the exemplary creationprocess of FIG. 7;

FIG. 9 is a flow diagram illustrating an exemplary process of gatheringInternet-based information using non-programming means;

FIG. 10 is a diagrammatical representation of an exemplary process ofthe non-programming development of rules associated with the voiceportal of FIG. 1;

FIG. 11 is an exemplary graphical user interface for the non-programmingdevelopment of rules associated with the voice portal of FIG. 1;

FIG. 12 is an exemplary graphical user interface window used in thenon-programming development of rules associated with the voice portal ofFIG. 1;

FIG. 13 is an expanded form of the graphical user interface window ofFIG. 12;

FIG. 14 is an exemplary graphical user interface search data editorwindow used in the non-programming development of rules associated withthe voice portal of FIG. 1;

FIG. 15 is an exemplary graphical user interface window used in thenon-programming development of rules associated with the voice portal ofFIG. 1;

FIG. 16 is an expanded form of the graphical user interface window ofFIG. 15;

FIG. 17 is an exemplary graphical user interface window used in thenon-programming development of rules associated with the voice portal ofFIG. 1;

FIG. 18 is an exemplary graphical user interface window for vendor formoptions used in the non-programming development of rules associated withthe voice portal of FIG. 1;

FIG. 19 is an exemplary graphical user interface window for the testingof a URL in the non-programming development of rules associated with thevoice portal of FIG. 1;

FIG. 20 is an exemplary graphical user interface window for theselection of patterns in the non-programming development of rulesassociated with the voice portal of FIG. 1;

FIG. 21 is an exemplary graphical user interface window used to identifypatterns for the detection of links on multiple pages during thenon-programming development of rules associated with the voice portal ofFIG. 1;

FIG. 22 is a diagrammatical representation of the hierarchical structureused in the programming of a spider;

FIG. 23 is an exemplary graphical user interface window for theprogramming of a spider used with the voice portal of FIG. 1;

FIG. 24 is an expanded form of the exemplary graphical user interfacewindow of FIG. 23;

FIG. 25 is a flow diagram illustrating an exemplary process of fusinginformation into a unified database of the voice portal of FIG. 1;

FIG. 26 is a flow diagram illustrating a second exemplary process offusing information into a unified database of the voice portal of FIG.1;

FIG. 27 is a diagrammatical representation of the creation of acanonical existant from two existants for more complete information on agiven item;

FIG. 28 is a diagrammatical representation of a first portion of anexemplary process of data isolation and transformation from an Internetsource to a user of the voice portal of FIG. 1;

FIG. 29 is a diagrammatical representation of a second portion of theexemplary process of FIG. 28 in which data is isolation and transformedfrom an Internet source to a user of the voice portal of FIG. 1; and

FIG. 30 is a flow diagram illustrating an exemplary operational flow ofthe voice portal of FIG. 1;

FIG. 31 is a flow diagram illustrating an exemplary operationalsubsystem of the flow diagram of FIG. 30;

FIG. 32 is a flow diagram illustrating a second exemplary operationalsubsystem of the flow diagram of FIG. 30;

FIG. 33 is a flow diagram illustrating a third exemplary operationalsubsystem of the flow diagram of FIG. 30;

FIG. 34 is a flow diagram illustrating an exemplary process of funnelinguser responses in the voice portal of FIG. 1 to determine a desired itemor service;

FIG. 35 is a flow diagram illustrating an exemplary process of carryingout a transaction using the voice portal of FIG. 1;

FIG. 36A is a flow diagram illustrating an exemplary process ofadvertising using the voice portal of FIG. 1;

FIG. 36B is a flow diagram illustrating a second exemplary process ofadvertising using the voice portal of FIG. 1;

FIG. 37 is a flow diagram illustrating an exemplary dialog map of thevoice portal of FIG. 1;

FIG. 38 is a flow diagram illustrating an exemplary subsystem of theexemplary dialog map of FIG. 37;

FIG. 39 is a flow diagram illustrating a second exemplary subsystem ofthe exemplary dialog map of FIG. 37;

FIG. 40 is a flow diagram illustrating a third exemplary subsystem ofthe exemplary dialog map of FIG. 37;

FIG. 41 is a flow diagram illustrating a fourth exemplary subsystem ofthe exemplary dialog map of FIG. 37;

FIG. 42 is a flow diagram illustrating a fifth exemplary subsystem ofthe exemplary dialog map of FIG. 37; and

FIG. 43 is a flow diagram illustrating a sixth exemplary subsystem ofthe exemplary dialog map of FIG. 37.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

A system and method for using voice over a telephone to access, process,and carry out transactions over the Internet are described. In thefollowing description, for purposes of explanation, numerous specificdetails are set forth in order to provide a thorough understanding ofthe present invention. It will be evident, however, to one skilled inthe art that the present invention may be practiced without thesespecific details. In other instances, well-known structures and devicesare shown in block diagram form in order to facilitate description ofthe preferred embodiments of the present invention.

One aspect of an exemplary embodiment of the present invention includesa method for executing a transaction related to an item or a serviceusing a telephone. The method includes providing information identifyingthe item or the service and providing a query as to a transaction to beperformed in which the transaction is related to the identified item orservice; and sending to a server system a request to execute thetransaction related to the identified item or service in response to auser answer. The transaction is executed without the user performing asingle action on a computer interface.

Advantageously, the system and method may include a voice-basedtelephone service which allows consumers to obtain pricing informationfrom, and place orders with, online retailers while they are shopping ina traditional store or anywhere telephone access is available.Advantageously, the speech user interface can ask a series of questionsthat allow the user to uniquely identify the product they are interestedin. These prompts can vary for different product types and can beconfigured at run-time. The user can respond by speaking an answer, withfallbacks to spelling and DTMF (“dual tone multi-frequency”, or thetouch tones on a telephone) when necessary. Further, the speech userinterface can interact with a virtual product database (consisting oflocally cached information and information retrieved at run-time fromthe Internet). Yet further, the speech user interface can return productpricing and availability information from various online retailers tothe consumer with minimal delay. Yet even further, the speech userinterface allows the user to order the product from their choice ofretailer.

Another aspect of the present invention is related to a system whichprovides voice access to Internet-based information and services. Yetanother aspect of the present invention is related to a system andmethod for determining if one web site has the same information asanother web site. Even yet another aspect of the present inventionrelates to a system and method for advertising using an Internet voiceportal. Still yet another aspect of the present invention relates to asystem and method for non-programming development of rules used in thetransformation of Internet-based information. Another aspect of thepresent invention is related to a system and method for funneling userresponses in an Internet voice portal system in order to determine adesired item. Another aspect of the present invention is related to asystem and method for the transformation and canonicalization ofsystematically structured data.

In one embodiment, a computer system is used which has a centralprocessing unit (CPU) that executes sequences of instructions containedin a memory. More specifically, execution of the sequences ofinstructions causes the CPU to perform steps, which are described below.The instructions may be loaded into a random access memory (RAM) forexecution by the CPU from a read-only memory (ROM), a mass storagedevice, or some other persistent storage. In other embodiments,hardwired circuitry may be used in place of, or in combination with,software instructions to implement the present invention. Thus, theembodiments described herein are not limited to any specific combinationof hardware circuitry and software, nor to any particular source for theinstructions executed by the computer system.

FIG. 1 illustrates a connection between a voice portal 10 and a network20. In an exemplary embodiment, network 20 is the Internet, a worldwidenetwork of computer networks that use the TCP/IP network protocols tofacilitate data transmission and exchange. In alternative embodiments,network 20 is any type of network, such as, a virtual private network(VPN). Network 20 preferably provides communication with HypertextMarkup Language (HTML) Web pages 30 and 40. Web pages 30 and 40 includea variety of data on a variety of Web servers. Network 20 also providescommunication with non-voice portal 50 which couples computers 52 and 54and a service 56 including a database 58 to network 20. Service 56 isany type of company, content or service provider with a connection tonetwork 20. Database 58 is a storage medium for data and may be anoptical, magnetic, or any other suitable storage medium.

Generally, voice portal 10 is implemented as a network of servers.Servers can be configured by software. Preferably, the servers include asignificant amount of read/write memory including disc drives and otherstorage. In general, users access voice portal 10 via telephones, suchas, a cell phone 12 or a standard telephone 14 by calling a telephonenumber (using the plain old telephone service (POTS)) which initiatescommunication between the telephones and voice portal 10. Alternatively,other types of telephone service can be utilized to communicate voice orvoice data to portal 10. The portal 10 can be connected to telephones 12and 14 via a variety of lines, networks, and stations. Advantageously,voice portal 10 provides for voice communication with the user. Voiceportal 10 allows the user access to information and services from webpages 30 and 40 as well as other sources available via network 20. Suchaccess is provided in a quick and efficient way by voice portal 10continually retrieving, organizing, and storing information from avariety of web sites and Internet services. Other user interfaceplatforms may also be provided for using voice portal 10. Such userinterface platforms include, for example, WAP (wireless applicationprotocol) and web interfaces.

FIG. 2 illustrates exemplary functional operations carried out by voiceportal 10. These functions may be carried out in any variety of ways,including any number of physical structures. In an exemplary embodiment,voice portal 10 includes a user interface 110, an advertising subsystem120, a customer management subsystem 130, an existant subsystem 140, afusion engine 150, an update engine 160, and a database 170.

User interface 110 coordinates voice communications between voice portal10 and the user. User interface 110 can be either via speech, via theInternet or “world wide web” (WWW), via a wireless application protocol(WAP) interface, or any other platform interface. In an exemplaryembodiment, user interface is speech oriented. In such a speech orientedembodiment, user interface 110 uses word-based automatic speechrecognition (ASR) for accepting user input wherever possible. Userinterface 110 can use a speech recognition software package, such as,Speech Works provided by Speech Works International of Boston, Mass. Forhigh rates of speech recognition, user interface 110 advantageouslyutilizes a funneling process which funnels user response to a set ofrecognizable answers. Funneling is described further with reference toFIG. 34. User interface 110 also uses spelling-based ASR for acceptinguser input when word-based ASR is not possible. Finally, user interface110 uses keypad entry for accepting user input only when advantageous touser. The key entry utilizes the keys on telephones 12 and 14 (FIG. 4).

In an exemplary embodiment, user interface 110 performs one or more ofthe following tasks: (1) Identify a user with a phone number and otheruser-specific information. (2) Start a new session for a given user on agiven platform. (3) Add a new interaction for a given user on a givenplatform. (4) Update a user's preference within the set of verticaldomains of interest available in voice portal 10. (5) Enable or disableuser preferences for that vertical domain of interest. (6) Update auser's expertise level either generally or within a specific vertical.(7) Update a user's demographic or personal information (as well ascredit card information). (8) Update a user's session state with userinterface-specific information. (9) Add a new credit card to thedatabase. (10) Update an existing credit card with new information. (11)Identify a credit card with the credit card type and number and check ifit is in the database already. (12) Set the list of vertical domainsavailable to the user and its order. (13) End a user's session normally.(14) Notify customer management subsystem 130 that the user's sessionabnormally terminated into some defined status (e.g., call dropped,session timeout). (15) Determine the most recent session of a user givena certain platform, such that it is possible to resume a session if asession was abnormally terminated (e.g., dropped call, sessiontime-out), and return the session state that was stored. User interface110 can perform additional functions related to identification, session,user, and payment protocols.

Advertising subsystem 120 coordinates activities related to theadvertisements to be presented to the user during a communicationsession. In an exemplary embodiment, advertising subsystem 120 includesadvertisements, such as, sponsored advertisements, advertisementstargeted to particular users, and permission-based advertisements whichare presented only after an affirmative request from the user. In anexemplary embodiment, advertising subsystem 120 provides one or more offunctions: (1) Choose an advertisement to play based on the user,session, location, content and item being explored. (2) Record that anadvertisement was played and if it was completed or not. (3) Record thata speak-through (i.e., as described below, an advertisement where theuser selectively chooses to hear more on an advertised subject) wasmade. (4) Store customer and session information within a bean so thatrepeated database calls are not needed. (5) Create a record for acompany that provides advertisements, and be able to identify one. (6)Create an advertisement and an advertisement's contract to be stored inthe database (as an advertisement may have different contracts for usageon the system). (7) Create a new sales employee or employer contact forAdvertising sales purposes. (8) Update an advertisement and/or thecontract of that advertisement. (9) Update an advertisement company tochange contact information and address information. (10) Update salesemployees and employer contacts. (11) Place/remove an advertisementin/from the active list. (12) Mark the advertisement's contract to becompleted or incomplete based on external information. (13) Display alist of active advertisements based on the advertisement type. (14)Display a list of advertisements relating to a company based on criteriaof either inactive, active, incomplete, complete or simply all theadvertisements. (15) Display a list of contracts relating to anadvertisement based on the above criteria. (16) Display a list ofcontracts relating to a sales' employee based on the above criteria.(17) Retrieve the completed listing of an employee, company,advertisement or advertisement contract by simply passing in a uniqueidentifier. (18) Search the database for near string matches ofemployee, company, advertisement and advertisement contract existants.(19) Keep track of the deliveries paid for a company on a specificcontract and be able to update the outstanding balance of the company.(20) Search the update logs to make sure that no data integrity errorsare present. (21) Create and modify a playlist needed to storeadvertisements for a specific genre.

A variety of different methods may be used to carry out each of theseoperations. Advertising operations are described further with referenceto FIG. 36. Advertising subsystem 120 can perform additional functionsrelated to identification, session, user, and payment protocols. Theadvertising techniques disclosed herein can also be used with aconventional personal computer (PC) interface web connection.

Customer management subsystem 130 coordinates the management ofinformation related to the user and the user's use of voice portal 10.In an exemplary embodiment, customer management subsystem 130 acquiresinformation on the user, such as, preferences and demographics, whichare selectively used by user interface 110, advertising subsystem 120,and other functions of voice portal 10. Customer management subsystem120 can perform additional functions related to identification, session,user, and payment protocols. Although subsystems 110, 120 and 130 aredescribed separately, the operations of each can be integrated in asingle unit without departing from the principles of the invention.

User Interface (UI) 110 and customer management subsystem 130 interactto provide for the selection of vertical domains and the access ofInternet-based information. Vertical domains categorize various fieldsor areas from which a user may select within voice portal 10. In orderfor UI 110 to communicate effectively with a user, certain preferencesand user facts must be ascertained and understood either passively oractively. Customer management subsystem 130 stores such information intodatabase 170. Alternatively, a separate customer database could maintainsuch information.

Customer management subsystem 130 acquires information needed todetermine customer preferences and facts from UI 110. UI 110 passes datainto customer management subsystem 130, which processes it, and thenrelays it to at least one database. Further, there are the updates ofpreferences in existant subsystem 140 for further parsing. Then,existant subsystem 140 passes information such as user preferences andfacts back to UI 110.

Advantageously, customer management subsystem 130 is modifiable andextensible without making response times appreciably longer. As such,the process to add new vertical domains to voice portal 110 is rapid andconsistent. The type of customer facts and demographics can never becompletely defined, as new vertical domains can always be added.

Customer management subsystem 130 records all transactions made by usersthat are subscribed and unsubscribed via a database. Customer managementsubsystem 130 also records items that the user locates in a formedhistory list and tracks the collections that the user looked at (on theweb site and through WAP devices).

Customer management subsystem 130 identifies subscribed customerswhenever possible, and as passively as possible. Thus, recognition ofcustomers preferably takes place via some sort of identification key,such as, for example, a telephone number and an ID (“PIN”) upon enteringa system. This identification preferably leads to certain preferencesassociated with the customer and experience level of a customer withineach set of preferences. Additionally, the system allows for anadditional level of identification (e.g. password identification) beforeauthorizing a purchase to be made against stored credit cardinformation.

Customer management subsystem 130 maintains, within each of the verticaldomains, a set of preferences to facilitate the user interactions viavoice portal 10. For example, in an exemplary embodiment, customermanagement subsystem 130 gathers information from the customer in orderto further help determine what type of advertising to give to thecustomer, and how to improve the customer's service. Customer managementsubsystem 130 maintains customer preferences appropriate to eachsupported domain and updates customer data from data sourcesdynamically. For example, in the Auctions domain of interest, currentbid status updated on user request. Voice portal 10 advantageouslypresents user data with currency appropriate to the domain. For example,in the Auctions domain of interest, bids are always current to withinseconds. In the e-commerce domain of interest, pricing information iscurrent when purchase price is presented.

Advantageously, customer management subsystem 130 provides reporting andanalysis to allow determination of which users are accessing whichservices. Further, customer management subsystem 130 provides reportingon session and transaction history by different demographic segment suchas, for example, determining which services are accessed by users from acertain income bracket, gender, or age group. Customer managementsubsystem 130 also provides reporting of relatedness based on actual use(e.g., the ability to report what other services are accessed by userswho are interested in movies).

In order to continually add value and transition with users from oneplatform to another (for example from the phone to the web, or from thephone to WAP), customer management subsystem 130 advantageously supportspersonalization features to improve customers experience with theservices. In addition to personalization, other sources of “stickiness”(customers “sticking” with the service in light of competition) includesthe support of community features such as networks of friends or offolks with common interests. Thus, customers tend to be more loyal tothe particular service provider if personalization features andcommunity features are included with customer management subsystem 130.

To support any adaptation of service (or advertising) to customerbehavior, customer management subsystem 130 advantageously tracks use ofservices. Further in the area of interface evaluation, typical userexplorations of interface hierarchies may help identify problem areas orvery useful areas, or correlated sets of sub-features in singlesessions. Another example of an important attribute the services ofvoice portal 10 is timing. For example, the use of a “barge-in” (where auser can interrupt with an answer before a list or prompt is finished)can signify a more advanced user and a string of barge-in selections toa single sub-tree repeatedly for a specific user may advantageously bedetected by customer management subsystem 130 and lead to an opportunityfor a short-cut—either a general one, or possibly a customer-specificone.

An aspect of “stickiness” is adaptation of services to a customer'spreferences. While this can include relatively simple features such ascustomer information retention in support of non-repetitive “sign-up” or“purchasing” data-entry requirements, it can also include preferencesfor navigation of particular sub-trees of interaction in differentfront-ends, and preferences for service/vendor ordering or selection. Asan example of vendor preference or ordering, a user may select a“preferred vendor,” allowing voice portal 10 to limit a list of vendorsfor a found product to two: cheapest and preferred.

Vertical preferences should be passively set based on user's actionsThat is, after requesting a particular attribute more than once, apassive preference would be set. Alternatively, preferences are dynamic,changing based on user's actions. Preferably, users are able to overrideall passive preferences, by setting or resetting them either throughvoice or web interfaces.

Customer management subsystem 130 can pull user preferences, such asstock information, weather preferences and (the like) from personalizedweb pages such as MyYahoo, and MyExcite. The personalized web pages canbe previously created by the user via a conventional Internetconnection. Alternatively, the personalized web pages can be built bycustomer management subsystem 130 in response to user voice commands.These pages can then be translated to be used with voice portal 10.General preferences can advantageously be used as a default preferenceif specific vertical preferences or current call preferences do notexist.

The following is a listing of exemplary vertical preference requirementsand their descriptions. Each preference is used differently throughouteach interface. In an exemplary embodiment, the only preference forweather is the weather for the location that the customer requests. Bydefault, the user's location is their ZIP code. The Most Commonly UsedLocation could be overridden by a current call location, if available.

In the Sports domain of interest, there are several differentpreferences to be looked at. First, favorite sports are an option.Certain sports scores, schedules, and standings can also be sent to theuser. For web sites, exclusivity can be used to not send advertisementsand information of certain sports. For example, one user may not want tohear information on hockey games, rather the user wants baseballinformation. Second, the preference granularity can be increased withcertain teams being preferred over others. In each of thesegranularities, it is possible that a most-recently used (MRU) list oflimited choices is used to determine the preference list. Besides typeof sport and team preferences, preferred events may be used.

In the Movies domain of interest, needed preferences include locality ofcustomer and locality of theaters, types of movies (e.g., thrillers,horror, action, etc. . . . ), ratings of movies (AA, G, R, etc. . . . ),and movies with a customer's favorite actors/actresses in the movies.Each of these preferences may be listed in an MRU list of limitedchoices.

In the Traffic domain of interest, the main preference used would bespecific routes that the customer would like to use to reach adestination, with an attribute being the time (the current being thedefault). Thus, an MRU list of limited routes could make up thepreference list of a customer.

In an exemplary embodiment, there is a two-level hierarchy ofpreferences for the Stocks domain of interest. First, there is apreference for a market list and second, within each market, there is apreference of which stocks and indices to look at. Again, an MRU list ofTBD choices of markets and stocks may be tabulated. Other verticaldomains of interest may include restaurants, concerts and live events,taxis, and airline reservations.

Referring still to FIG. 2, existant subsystem 140 coordinates access byuser interface 110, advertising subsystem 120, customer managementsubsystem 130, fusion engine 150, and update engine 160 to database 170.Existant subsystem 140 manages the creation, adaptation, andmanipulation of the data structures contained in database 170. Datacontained in database 170 is gathered from Internet sources by updateengine 160. In an exemplary embodiment, the data structure used indatabase 190 is based on a hierarchy of “existants” or things and theirrelationships and associations with each other. Advantageously, theability to replicate and modify information in database 170 is moreeasily done because database 170 interacts only with existant subsystem140. Existants and their creation are described further with referenceto FIGS. 4-10. Specifically, an exemplary data structure model forexistants is described with reference to FIGS. 4-6 although variousother structures for existants can be utilized. Creation and updating ofexistants are described with reference to FIGS. 7-10.

Fusion engine 150 determines whether two existants are the same and, ifso, fuses the existants to form a third canonical existant. As such,fusion engine 150 establishes whether information gathered from onesource is related or the same as information gathered from anothersource. Functions of fusion engine 150 are described further withreference to FIGS. 25, 26, and 27.

Update engine 160 retrieves information from the Internet to updateinformation and attributes contained in database 170. In an exemplaryembodiment, update engine 160 utilizes “spiders” which retrieveinformation from the Internet in order to update information in database170. Operations of update engine 160 are described further withreference to FIGS. 7 and 8.

Database 170 stores information used by voice portal 10, such as,customer data, advertising information, and product and servicesinformation. Information in database 170 is stored into existants,existant attributes, existant relationships, and existant associations.What existants are, how they are formed, how they are related to eachother, and how they relate to the functionalities of voice portal 10 aredescribed further below. In alternative embodiments, multiple databasesmay be used for specific types of information, such as, customer data,advertising information, and operations records.

FIG. 3 illustrates an exemplary physical layout of voice portal 10.These physical structures are by way of example only. Other structuresmay be used in combination with or in lieu of those shown. In anexemplary embodiment, voice portal 10 includes front end servers 210, afront-to-back network 220, back end servers 230, and a back-end network240. Users communicate via telephone with one of the front end servers210, which is coupled to back end servers 230 by front-to-back network220.

In an exemplary embodiment, back end servers 230 include a proxy manager245, proxies 250, beans 260, and a database 270. Proxy managers 245receive requests for information from one of front end servers 210 viafront-to-back network 220. Proxy managers 245 communicate via back endnetwork 240 to determine work load levels at each of proxy managers 245.Once an appropriate proxy manager 245 is determined, the appropriateproxy manager 245 pulls a free proxy from a pool of free proxies 250 andassigns the proxy to a bean 260. Bean 260 is associated with database270 in order to retrieve information, insert information, searchexistants or existant relationships, or perform any other functionpossible with database 270.

The virtual database structure described with reference to FIG. 3 isdesigned to deliver information garnered from Internet 20 to users ofvoice portal 10 in a timely and highly utilitarian way. People need anduse information in a variety of settings and ways, and, advantageously,voice portal 10 supports this on a variety of platforms including, butnot limited to, the phone (e.g., voice, WAP and both), the web, andportable connected computing devices (e.g., Palm® OS device, WinCE®device, RIM pager).

Back end servers 230 include a database service support with a varietyof features, including data collection and fusion. Data collectionincludes the amassing of data from Internet sources at regular intervalsto be scheduled for particular item types and/or sites, as describedwith reference to FIGS. 7 and 8. Voice portal 10 detects changes to datasource sites and notifies the appropriate site rule manager, asdescribed with reference to FIGS. 9 and 10. Voice portal 10 alsosupports non-expert definition of data extraction for data sources, asalso described with reference to FIGS. 9 and 10.

In the process of “fusion,” voice portal 10 identifies identical itemsfrom different Internet vendors. During the fusion process, voice portal10 retains meta data about the source of all information. Meta dataincludes data about data. For example, meta data may document data aboutdata elements or attributes, (name, size, data type, etc.) and dataabout records or data structures (length, fields, columns, etc.) anddata about data (where it is located, how it is associated, ownership,etc.). Further, voice portal 10 supports interactive clarification offusion decisions or non-decisions by non-experts in cases wherecertainty cannot be determined automatically. Voice portal 10 alsosupports additions of new data types and data elements, without codechange. Even further, voice portal 10 supports domain-specific conceptsof relatedness that are identified through market research, trial, andopportunity. For example, in the e-commerce domain of interest, cheaper,better, often-bought-with, and most-popular are important relatednessconcepts. In the movies domain of interest, related movies and products,and best movies in a category, most popular, best by reviewer, and cast,are important relatedness concepts. Voice portal 10 collects and retainsrelated information necessary to provide additional detail about an item(e.g., product descriptions). The operation and functionalities offusion are further described with reference to FIGS. 25-27.

FIG. 4 illustrates an exemplary data structure model 300 used bydatabase 170 of voice portal 10 in which “existants” (or things) aregiven attributes, associations, and relationships. An “inheritance”relationship between existants is depicted by a solid line with atriangle head. An “association” relationship between existants isdepicted by a dashed line with an open head arrow. For example of aninheritance relationship, in the data structure model 300, a block 310is an “event”. An “event” is an “existant” or thing, illustrated by atriangle headed arrow 315 to a block 320. Similarly, a “movie showing”(block 330) is an “event” (block 310), illustrated by a triangle headedarrow 335. For example of an association relationship, events areassociated with “venues”, as illustrated by an open headed arrow 345 toa block 340. Similarly, a movie showing (block 330) is associated with a“movie package” (block 350), as shown by an open headed arrow 355.Events can also be sporting events, dramas, concerts, comedy shows,fireworks presentations, dancing performances, or any other activity.

Data structure model 300 includes more existants, associations, andrelationships which are illustrated in FIG. 4, but not described here.Further, data structure model 300 may include more existants,associations, and relationships which are not included in theillustration. FIG. 4 is for illustration only.

Referring now to FIG. 5, an exemplary data structure model 400 isillustrated to show the object oriented relationships between a user orcustomer object and the different vertical classes. Depictions ofinheritance and association relationships are the same as in thedepiction of data structure model 300 described with reference to FIG.4. In an exemplary embodiment, user information arranged in datastructure model 400 is contained in database 170. However, inalternative embodiments, such user information may be contained in aseparate customer database.

The customer is an Existant (Customer Existant Block 402) and is adirect descendent of the top level, or the “Existant” existant, withinthe hierarchy, and thus inherits all of its properties and methods. Thereason behind this structure is that database 170 and its methods arealready created and the structure allows reuse of code.

The customer object contains various pieces of information. The genericPreferences class contains information on preferences, such as,“Traffic”, “Weather”, and “Movies”. Each time a customer entered a newand different vertical domain of interest, an instance of thePreferences object to the vertical domain's name is created withpreference data inserted. If the vertical domain already exists, thenthe object is modified with the updated information.

Session class records the information directly about a user's session(session block 404). The session may be a call, a search through thewebsite, or a call using the WAP. Data, such as, time of day andduration are general attributes but analysis on whether the user made acall from a landline or cell phone is specific to phone sessions. Thistype of data is useful for determining to whom voice portal 10 directsmarketing (for advertising purposes), and improving both performance andservice. As well, a customer object has a link to each of these sessionobjects to determine what was the last session on that platform (in casea user terminated the session and would like to reconnect at thatspecific time).

A Phone Session block 408 records information relating to acommunication session where a telephone is used to communicate withportal 10. Phone Session block 408 includes information, such as, thecurrent level of interaction, the current domain of interest, the typeof interface platform (e.g., WWW, WAP, ASR), and previous levelsvisited. Advantageously, Phone Session block 408 allows the user torejoin a session where he or she left off in a previous session or froman interrupted session. Other existant blocks, such as, credit card infoexistant, location existant, and preferences existant contain associatedattributes and record information as needed.

The Expertise class (Expertise block 406) serves the purpose ofmaintaining different levels of usability (generically, and fordifferent preferences) across different platforms (i.e. Phone, WAP,WWW). The Customer has links to each of these class instances. These arenot included in the Preferences class since preferences can crossplatforms and user's capabilities cannot.

FIG. 6 illustrates an exemplary data structure model 450 used bydatabase 170 of voice portal 110 for information related to advertising.Depictions of inheritance and association relationships are the same asin the depiction of data structure model 300 in FIG. 4. In an exemplaryembodiment, advertising information arranged in data structure model 450is contained in database 170. However, in alternative embodiments, suchadvertising information may be contained in a separate advertisingdatabase.

Advantageously, data structure models 300, 400 and 450 provide for acontinually expanding arrangement of information on existants,associations, and relationships. Furthermore, models 300, 400, and 450allow for the creation of new vertical domains of interest quickly andwithout changing previously entered information. For example, model 300includes information related to events, such as, movies and concerts aswell as commodities, such as, books, toys, and electronics. Any event,such as, ballets can easily be added as an existant with an inheritancerelationship with “event” and appropriate association relationships.Similarly, any commodity, such as, a vehicle can easily be added as anexistant with an inheritance relationship to “manufactured item” andappropriate association relationships. The dynamic nature and expansivecapabilities of data structure models 300, 400, and 450 allow voiceportal 10 the advantage of being a unitary voice portal to a wide rangeof Internet-based information and services.

FIG. 7 illustrates a flow diagram 700 of an exemplary creation processof an existant, such as, existants shown in exemplary data structuremodel 300 (FIG. 4), data structure model 400 (FIG. 5), and datastructure model 450 (FIG. 6). In a step 710, a web page on the Internetis found. In an exemplary embodiment, a spider is used to findparticular web pages relating to a pre-determined category of items. Aspider is a conventionally known program that automatically explores theWorld Wide Web (WWW) by retrieving a document and recursively retrievingsome or all of the documents that are referenced in it. In contrast, anormal web browser operated by human does not automatically follow linksother than in line images and URL redirection. After step 710 isperformed, a step 720 is performed in which information is identified onthe found web pages by using a chosen form which overlays the page tofilter out particular information. After step 720, a step 730 isperformed in which rules are used to identify characteristic informationor attributes from the information retrieved by the form overlay in step720. Characteristic information or attributes define what the existantis. Rules define the organization of existant attributes. For example, amovie existant may include the attributes of a title, a director, acast, a release year, and a synopsis.

After step 730 is performed, a step 740 is performed in which attributesare organized within the existant and the existant is stored in database170. Preferably, the organization and arrangement of attributes withinthe existant are structured by pre-defined rules.

FIG. 8 illustrates the exemplary creation process of existants asdescribed with reference to FIG. 7. A spider 810 traverses Internet 20for information stored on a wide variety of different web pages.Information retrieved by spider 810 is organized and arranged accordingto rules 820 in order to place information in a data structure 830. Inan exemplary embodiment, spider 810 retrieves information from Internet20 regarding movies. For example, spider 810 may traverse the IMDB website and retrieve information regarding the title, director, cast, yearof release, and running time for a particular movie. Once movieinformation is stored in data structure 830, data structure 830 isapplied to a lexical table 840. Lexical table 840 organizes attributescontained in data structure 830 and places the information in threecolumns. In an exemplary embodiment, the first column of lexical table840 includes the original data, the second column includes the originaldata in a normalized and tagged format, and the third column includesthe data in a searchable and mashed format. Lexical table 840 and datastructure 830 are contained within memory structures in database 170.

By way of an example, if spider 810 traverses Internet 20 forinformation related to the movie “Raiders of the Lost Ark,” dataretrieved from Internet 20 will be applied against a rule correspondingto movies and placed in data structure 830. Such a rule for movies mayinclude title, director, cast, and release year, all of which areattributes of movies. In this example, the title would be “Raiders ofthe Lost Ark,” the director would be “Steven Spielberg,” the cast wouldbe “Harrison Ford and Karen Allen,” the year would be “1981,” and therunning time would be “115 minutes.” As such, lexical table 840 wouldcontain the title in its original format: “Raiders of the Lost Ark,” thedata in normalized and tagged format: <title> Raiders of the LostArk</title>, and in searchable mashed format: RaidersLostArk, withoutany spaces or identifying articles (e.g., the, a, an).

FIG. 9 illustrates a flow diagram 900 which depicts an exemplary processof gathering Internet-based information using non-programming means. Ina step 910, a search page is found and patterns are used to isolate thearea on the page containing relevant information. After step 910 isperformed, a step 920 is performed in which an appropriate form is foundand special routines are invoked to extract actual data and information.After step 920, a step 930 is performed in which multiple pages withrelated information is found given a particular page. Apart from thedata specific patterns, there is an area pattern that defines where dataspecific patterns operate in the particular page. After step 930 isperformed, a step 940 is performed in which links to more listings ofproducts or services on the multiple pages are found. In an exemplaryembodiment, a prediction routine is used to compute actual patterns ofproduct listings from code samples.

In general, the prediction routine computes a pattern from a desiredoutput given by a rule writer. Advantageously, the pattern predictionroutine speeds up production because a rule writer simply has to pastefrom the HTML code the text fragments that he or she wants extracted,without having to develop the patterns to make that happen. The inputfields currently used to write the patterns are used to insert thisdata.

By way of example, a prediction routine develops a pattern for Authordata for web pages giving data on books by first having the rule writercopy a sample author name on the web page into the Author field. Thealgorithm then matches the sample data to its location on the web page.Characters or tags proximate the matched data are identified as a“prefix” and “suffix”. Prefixes are characters before the matched dataand suffixes are characters after the matched data. The prefix andsuffix are used to construct a pattern.

The constructed pattern is applied to the web page to match againstother data. If the constructed pattern picks up data which is not equalto the desired result, then the prefix and suffix used to develop thepattern are added to, for a more complete and accurate pattern. Thisprocedure repeats to improve the pattern.

To further clarify this example, take the following HTML code from a webpage giving product data on books:

<html> <title>Programming Perl</title> written by <b>Larry Wall</b></html> <html> <title>Learning Perl (<b>2nd edition</b>)</title> writtenby <b>Randal Schwartz</b> </html>

The rule writer dumps “Larry Wall” in the Author field to indicate thatthis is the data to extract for Author.

The pattern prediction algorithm roughly works as follows:

n = 1; repeat  $page =~m / ({.}n) Larry \s+Wall ({.}n) /x;  $prefix =$1;  $suffix = $2;  $page =~m / $prefix (.*?) $suffix /x;  n = n + 1;until ($1 eq <desired_data>);Starting with n=1 on the first page, the algorithm matches “>LarryWall<” which means that $prefix gets value “>” and $suffix gets value“<”. Next, the pattern prediction algorithm builds the pattern “>(.*?)<”using the values for $1 and $2 it got from the first step. Matching thispattern against the web page results in “>Programming Perl<” which isnot equal to the desired result “Larry Wall”. Therefore, n isincremented to n=2 and the pattern is refined to include anothercharacter in the prefix and suffix. Matching the web page with “({.}2)Larry\s+Wall ({.}2)” results in “b>Larry Wall</” which means that$prefix gets value “b>” and $suffix gets value “</”. Next, the patternprediction algorithm builds the pattern “b>(.*?)</” using the values for$1 and $2 it got from the first step. Matching this pattern against theweb page results in “Larry Wall”, the desired output.

Now, as the rule writer steps through web pages to apply the samepattern on different pages, he or she discovers that the patternmatches: “2nd edition” on the page about book “Learning Perl”. The rulewriter then improves the algorithm by giving a second example of adesired result, (i.e., he or she dumps “Randal Schwartz” into GUI inputfield), which triggers the pattern prediction algorithm to furtherincrement n until a pattern enforcing a “y” before the <b> is created.The algorithm may perform several iterations, depending on thecomplexity of the web data data and the pattern needed.

After step 940 is performed, a step 950 is performed in which a vendorspecific data extraction file is generated. In an exemplary embodiment,a routine is used that computes relevant URLs from code samples.Alternatively, the routine that computes URLs can be passed as a form.After step 950 is performed, a step 960 is performed in which a cache iscreated. After step 960 is performed, a step 970 is performed in whichpatterns for extraction of product data are created. In a preferredembodiment, a regression test mechanism supports editing the specialroutines.

FIG. 10 illustrates an exemplary process of the non-programmingdevelopment of rules associated voice portal 10. In the exemplaryprocess, a rule writer from a set of rule writers 1010 accesses theWorld Wide Web (“WWW”) 1020 in order to access information from any oneof data source 1030, data source 1035, data source 1040, or any otherdata source connected to WWW 1020. Data retrieved from a data source isplaced into a data structure utilizing a data organizing tool 1025. Rulewriters 1010 use data organizing tool 1025 to apply one of a multitudeof possible forms to “pages” of information available via WWW 1020. Suchforms provide indications for the location of relevant information onthe page and labeled with some distinctive tag. For example, a pageprovided on WWW 1020 may include a data input box in the upper left handcorner of the page. Further, relevant information on a part or servicemay be located after a HTML tag, such as, “<title>” for the title of abook.

It should be noted that as used herein the term “page” includes a userinterface screen or similar arrangement which can be viewed by a user ofthe diagnostic system, such as screens providing graphical or textualrepresentations of data, messages, reports and so forth. Moreover, suchpages may be defined by a markup language or a programming language suchas Java, perl, java script, or any other suitable language.

Using the form selected by rule writers 1010 from data organizing tool1025, data from data sources is organized into a data structure 1045,data structure 1050, data structure 1055, or any similar structure formaintaining information. Data structures 1045, 1050, and 1055 may becompared, fused, or utilized in the formation of a unified datastructure 1060. Unified data structure 1060 is stored in a database1070.

Advantageously, the exemplary process illustrated in FIG. 10 allowsnon-expert rule writers 1010 to select from a variety of forms providedby data organizing tool 1025 to use in the retrieval of information fromparticular web sites available via WWW 1020. As such, data contained onweb pages from data sources 1030, 1035, and 1040 can continually beupdated to database 1070 with the form selected by rule writers 1010using data organizing tool 1025. As information contained in datastructures 1045, 1050, and 1055 are compared for accuracy, dataorganizing tool 1025 detects when web pages have changed the format orarrangement of data on their corresponding web page.

FIGS. 11-24 illustrate an exemplary process of creating a new rule.Further, FIGS. 11-24 illustrate possible interactions between a rulewriter and data organizing tool 1025 (FIG. 10). One exemplary rule isbased off an existing rule: the Amazon.com book products. The stepstaken in constructing this rule are similar to the steps taken inconstructing any other rule.

FIG. 11 illustrates a graphical user interface (GUI) 1110 which is usedto initiate the creation of rules 820 (FIG. 8). GUI 1110 includes avendor window 1120, a spider selection window 1130, a query window 1140,a status window 1150, a search box area 1160, and a code window 1197.Search box area 1160 includes a slider bar 1170, right set of arrows1180, left set of arrows 1190, and a search window 1195.

To start a new data source, a rule writer enters the data source (e.g.,Amazon Book) in a vendor window 1120. The rule writer presses ‘Enter’and clicks the ‘New’ button. After this action is performed, a graphicaluser interface (GUI) 1200 illustrated in FIG. 12 is shown. The rulewriter clicks on the ‘Done’ button after confirming that the data sourceis listed correctly. Next, graphical user interface (GUI) 1300illustrated in FIG. 13 is shown. A URL is displayed corresponding to theselected vendor name. The rule writer is asked to confirm the correctURL. In the example of Amazon Book, the URL http://www.AmazonBook.comappears in a window of GUI 1300. However, the URL link should readhttp://www.Amazon.com. The rule writer corrects the URL and clicks onthe “done” button.

Referring again now to FIG. 11, the rule writer selects the type ofquery that is desired. First, the rule writer selects query window 1140and chooses from a list of potential queries. For example, “bookpackage” maybe a possible query for the book vertical domain ofinterest. This search is started when the rule writer clicks on the“SDE” (search data editor) button in query window 1140. The SDE buttoninvokes a search data editor, which provides a graphical user interface(GUI) 1400 illustrated in FIG. 14. GUI 1400 shows a list of attributesuseable in a search for the particular item of interest. For example,where books are being searched, attributes such as ISBN or UPC areshown. Where searches are for other items, attributes are listed whichcorrespond to that item. A search for “Movie Showings” results withattributes listed, such as, Movie Package, time, and showing date (seeblock 330 described with reference to FIG. 4).

The rule writer types a ISBN number into the corresponding data box andclicks ‘Done’. Buttons 1430 in GUI 1400 advantageously allow the rulewriter to save different search criteria during different searches. Oncethe search criteria is entered, the rule writer clicks ‘Done’ andbecause no rules have been defined for the particular data source (i.e.,Amazon Book), a graphical user interface (GUI) 1500 illustrated in FIG.15 appears. GUI 1500 asks whether the rule writer wants to add a newrule or change the search data. In this example, the rule writer clickson the “add” button and GUI 1500 expands to become graphical userinterface (GUI) 1600 illustrated in FIG. 16.

Referring now to FIG. 16, the rule writer confirms that the correct typeof query is highlighted. In this example, ISBN is highlighted and therule writer clicks on the “yes” button. A graphical user interface (GUI)1700 illustrated in FIG. 17 appears to instruct the rule writer that thehome page of Amazon Book is loaded into the netscape browser. The rulewriter is instructed to browse the web page associated to the ISBN rule.Once the search page is loaded into the Internet browser, the rulewriter clicks the “done” button.

A graphical user interface (GUI) 1800 illustrated in FIG. 18 shows aform option to be selected by the rule writer. If the form is correct,the rule writer clicks the “done” button. If the form listed does notgive the rule writer the choices required, the rule writer clicks on the“next” button to see other forms on the page. Once a page that matchesis found, a graphical user interface (GUI) 1900 illustrated in FIG. 19is displayed.

Data organizing tool 1025 (FIG. 10) displays the resulting page in theInternet browser. If the page is correct, the rule writer clicks on“okay” on GUI 1900. A graphical user interface (GUI) 2000 illustrated inFIG. 20 appears and asks how to detect single items on the page if thesearch matches on multiple items. GUI 2000 is also used to indicatewhere to find the URL to get details about the queried item. If only asingle item was found, the rule writer clicks the “defer” button becausenot enough information is present to build the regular expression. Ifmultiple items are found, a regular expression is entered into a datawindow 2010. For example, an author search may return multiple itemsbecause a single author may have written several books. In other cases,even if the query only matches one item, it may be necessary to followan additional URL link to get the information.

A graphical user interface (GUI) 2100 illustrated in FIG. 21 appearsnext and is used to detect multiple product pages. If the rule writergoes directly to the item searched for, there is no need for informationto build the regular expression. Referring again to FIG. 11, code window1197 is filled with HTML code from the retrieved page. At this point,the rule writer is ready to specify attributes. Attributes are specifiedby entering a regular expression into the box next to their name. Theregular expression must specify one substring in it (using parenthesis)as a result of the expression. For example, the regular expression:“this (all) matches” would return “all” as its result (assuming that theregular expression was able to match). For example, determining thepattern used to find the title of a book requires that the rule writertype in the title of the book into search window 1195. A variety of HTMLsignals may be used. The “\s*” is required to indicate possible blankspace between words. The first match of the search string entered insearch window 1195 will highlight the first match found in the HTMLcode. For example, a title may be found inside a pair of <title> tagswith some extra information. An exemplary attribute for title of a bookmay be “<title>([̂<]*)</title>”. Once the attribute is entered, allmatches to the attribute are found.

Referring again to FIG. 14, search data editor 1400 consists of a formwhich can be used to assign values to type dependent attributes. Thestatus window indicates what data organizing tool 1025 is doing. In anexemplary embodiment, the status states are idle, running the query overthe Internet, and using the cache. Query window 1140 allows the rulewriter to set the type of query desired for the data source in questionas well as set the search criteria by using the SDE button.

Spider selection window 1130 allows the rule writer to set the spider tobe used if not doing a query search. In an exemplary embodiment, thepossible spider types are full, incremental, special, and reference. Afull spider takes all items that match the chosen type. Incrementalspiders are usually used to pick up updates of data from Internet datasources. Special spiders are usually used to pick up something that thesite has that is special, such as, best sellers for books. Referencespiders are usually used to confirm that the site is still up and rulesare working.

Vendor window 1120 allows the rule writer to set the data source to workon. Search window 1195 allows the rule writer to keep text to besearched for in the HTML code. In code window 1197, there is a cursor toindicate position of text entry. Left set of arrows 1190 includes afirst number, which is the starting point of where the search will runwhen running from the cache. The second number indicates the totalnumber of pages in the cache. The set of arrows in this window controlsthe page to start from when the rule writer runs from the cache. Rightset of arrows 1180 includes arrows to scroll through pages retrieved.

Spiders are similar to queries but they are called when no other rulesare applicable. Spiders are responsible for gathering information onevery object in the web site that matches the specified type. The spiderconsists of several nested loops, each designed to go one level deeperinto the hierarchy. Referring now to FIG. 22, an exemplary spiderhierarchy 2200 is shown for a book spider where a level 2210 is thestart page, a level 2220 represents book category pages, level 2230represents book sub-category pages, and a level 2240 represents bookpages.

Referring now to FIG. 23, a graphical user interface (GUI) 2300 is usedto retrieve the URL of a page to be associated with a spider rule. Aspider depth slide rule allows the rule writer to tell data organizingtool 1025 how many links down it takes to get to the actual productpage. The upper bound slide rule allows the rule writer to specify alimit on how many items to spider. Once the URL is selected and a spiderdepth and an upper bound is selected, the rule writer clicks on the“done” button. A graphical user interface (GUI) 2400 illustrated in FIG.24 is shown. The rule writer enters search patterns for the spider touse in a similar fashion to the search patterns entered for a querydescribed with reference to FIG. 11. Once the pattern is entered, therule writer clicks on the “build” button and the spider will start torun.

Advantageously, the graphical user interfaces shown and described withreference to FIGS. 11-24 allow non-expert rule writers to perform datasearches and create forms of rules for information retrieval. Once theforms are created, the forms can be frequently used to gather updatedinformation. Further, forms are helpful to retrieving large amounts ofinformation available at a vendor's web site using a common formcorresponding to the arrangement and display of information by thevendor on the web site. Advantageously, forms of rule creation bynon-experts provides for lower costs to update information available atweb sites. Further, the forms advantageously automate accurate retrievalof Internet-based information.

FIG. 25 illustrates an exemplary process of fusing information in adatabase. In exemplary embodiment illustrated by FIG. 25, a flowchart2500 depicts a simplistic fusion process, or “quick fusion”, executed byfusion engine 150 (FIG. 2). In a step 2510, update engine 160 receivesinformation from network 20 and places the information in an existantdata structure in database 170 via existant subsystem 140. Fusion engine150 has access to existants from update engine 160 via existantsubsystem 140, which accesses database 170. After step 2510 isperformed, a step 2515 is performed in which fusion engine 150 gathersexact fusion attributes from an attribute definition table correspondingto the existant retrieved in step 2510. After step 2515 is performed, astep 2512 is performed in which fusion engine 150 executes a “mash” ofeach fusion attribute from existence retrieved from database 170 into aneasily comparable form. In an exemplary embodiment, a “mash” formremoves spaces, prepositions, and other non-essential words.Advantageously, a “mashed” format provides for quick searchingcapabilities.

After step 2520 is performed, a step 2525 is performed in which fusionengine 150 formulates a database query in which the data source is setto “same” and the status is set to “canonical”. This query is intendedto find an already existing canonical existant from the same data sourcefile which matches the current information. After step 2525 isperformed, a step 2530 is performed in which a decision is made whethera match in database 170 is found. If a match in database 170 is founddue to the query of step 2525, a step 2535 is performed in which theexistant contained in database 170 is updated.

If no match in database 170 is found from the query of step 2525, a step2540 is performed in which the query of step 2525 is reformulated andthe data source is set to “same” and the status is set to“non-canonical”. This query is intended to find an already existingexistant from the same data source file which matches the currentinformation. After step 2540, a step 2545 is performed in which adecision is made whether a match is found in database 170 from thereformulated query of step 2540. If a match is found, a step 2550 isperformed in which the existant is in database 170 is updated.

If no match is found in database 170, a step 2555 is performed in whichthe query is reformulated with the data source being set to “any” andthe status is set to “canonical”. This query is intended to find analready existing canonical existant from any data source which matchesthe current information. After step 2555, a step 2560 is performed inwhich a determination is made whether a match in database 170 is found.If no match in database 170 is found, a step 2565 is performed in whichan existant is added to database 170.

If a match in database 170 is found or after step 2550 is performed, astep 2570 is performed in which a determination is made whether thematch is a system existant. If the match is a system existant, a step2575 is performed in which the system existant is updated. If the matchis not a system existant, a step 2580 is performed in which a canonicalsystem existant is formed. After step 2580 is performed, a step 2585 isperformed in which the existant is added to database 170. After step2585, a step 2590 is performed in which the fusion tables are updated.

Advantageously, the exemplary process of fusing information in adatabase illustrated by FIG. 25 provides for the comparison ofinformation on multiple web sites. As such, a determination can be madeif one web site contains the same information as another web site.Furthermore, information contained in database 170 of voice portal 10can continually add information, relationships, and associations ofinformation from Internet-based sources, which provides for greaterusability of information retrieved from data sources.

FIG. 26 illustrates a flow chart 2600 depicting steps taken in anexemplary process of fusion. In the exemplary process described withreference to FIG. 26, a fusion process is shown which is morecomprehensive than the fusion process depicted in flow chart 2500described with reference to FIG. 25. In a step 2610, fusion engine 150reads an attributes definition table from database 170. After step 2610is performed, a step 2615 is performed in which fusion engine 150 readsa fusion control language file for each existant type requiring advancedfusion. After step 2615 is performed, a step 2620 is performed in whichfusion engine 150 compiles fusion files into an intermediate computercode. After step 2620 is performed, a step 2625 is performed in whichfusion engine 150 holds previously fused existants into memory. Afterstep 2625, a step 2630 is performed in which fusion engine collectsattributes into equivalent sets. After step 2630, a step 2635 isperformed in which a decision is made whether an attribute is textual.If fusion engine 150 determines that the attribute is not textual, astep 2640 is performed in which the values are indexed. If fusion engine150 determines attribute is textual, a step 2645 is performed in whichfusion engine 150 indexes substring occurrences in the attribute.

After step 2645, a step 2650 is performed in which fusion engine 150determines whether the text is structured. If the text is determined tonot be structured, a step 2670 is performed. If the text is determinedto be structured, fusion engine 150 identifies location and isolatedstructural segment of the text in a step 2655. After step 2655, a step2660 is performed in which fusion engine 150 parses isolated parts andidentifies semantic information. After step 2660, a step 2665 isperformed in which fusion engine 150 indexes semantic information. Afterstep 2665, step 2670 is performed in which fusion engine 150 executesvalidity checks to verify the integrity of database 170. After step2670, a step 2675 is performed in which fusion engine 150 retrieves theexistant to be fused.

After step 2675, a step 2680 is performed in which fusion engine 150activates fusion criteria and matching programs for correspondingexistant types. Fusion criteria and matching programs involve the use ofexistant rules which are established as described with reference to FIG.10. After step 2680, a step 2685 is performed in which fusion engine 150executes the first fusion rule from the fusion criteria and matchingprograms and returns all matches. After step 2685, a step 2690 isperformed in which a decision is made whether an acceptable match hasbeen found. In an exemplary embodiment, an acceptable match is one inwhich a predetermined percentage (e.g., 70%) of attributes are common.In an alternative embodiment, an acceptable match is one in which allattributes which have values that are the same. If an acceptable matchhas been found, a step 2697 is performed in which a fusion engine 150fuses existants together. If an acceptable match is not found, a step2691 is performed in which the next fusion rule is executed and allmatches are returned.

After step 2691, step 2692 is performed in which a determination is madewhether an acceptable match is found. If an acceptable match is found,step 2697 is performed in which fusion engine 150 fuses existantstogether. Fusion of existants includes the creation of a new existantwhich is associated with the existants to be fused and contains allinformation therein. If an acceptable match is not found, a step 2693 isperformed in which a determination is made whether the last rule hasbeen tested. If the last rule has not been tested, step 2691 isperformed again. If the last rule has been tested, a step 2694 isperformed in which fusion engine 150 determines whether there are strongpartial matches. In an exemplary embodiment, a strong partial match isone in which a match is within a certain percentage, such as, 70%. Ifstrong partial matches exist, a step 2698 is performed in which adeference is made to human examination. If partial matches are notfound, a step 2695 is performed in which fusion engine 150 rejects thefusion creation, and a step 2699 is performed in which a new existant iscreated.

Advantageously, the exemplary process of fusing information in adatabase illustrated in FIG. 26 provides for the automatic comparison ofinformation from the same or different data sources. As such,information contained in database 170 can continually be updated andgiven added relatedness to information from other data sources. Further,fusion allows for the compilation of a more complete and robust unifieddatabase than the millions of databases individually available on theInternet.

FIG. 27 illustrates an exemplary process of creating a canonical datastructure from two data structures. A data file 2700 is identified by aunique identification number and contains a first data file 2710, asecond data file 2720, and a canonical data file 2730. In an exemplaryembodiment, first data file 2710 contains information relating toparticular movie retrieved from the IMDB (“Internet Movie Data Base”)website (http://www.IMDB.com). Second data file 2720 includes movieinformation for a particular movie obtained from the Reel.com website.In the example illustrated by FIG. 27, data file 2710 includes a title“Boys of Arizona,” the director “Wiltz,” the release year “1997,” and asynopsis “great movie.” Similarly, data file 2720 includes a title “TheBoys of Arizona,” the director “Bob Wiltz,” the release year “1998,” anda synopsis which is blank.

During the process of creating a canonical data file, a rules file 2740is introduced which contains rules for a particular type of information.In the example shown in FIG. 27, rules file 2740 contains informationrelating to the attributes of movies. By application of rules 2740, achronicle data file 2730 is created by taking the most complete titlefrom data file 27 and data file 2720, which is the title from data file2710: “The Boys of Arizona.” Director information is obtained from datafile 2720 because it is more complete than data file 2710 because itcontains the director's first and last name. A conflict exists as to therelease year listed by data file 2710 and that listed by data file 2720.Based on prior information, the conflict is resolved to indicate thatdata file 2720 has a more correct release year. Chronicle data file 2730include the synopsis data file 2710 because the synopsis of data file2720 is blank.

Advantageously, the process of creating chronicle data files describedwith reference to FIG. 27 creates data files with more complete andaccurate information. Furthermore, the process allows comparison ofinformation between multiple websites. Even further, the process ofcreation of chronicle data files allows the increase in relatedness andassociation relationships among data files.

FIG. 28 illustrates a functional diagram 2800 of the operations carriedout during isolation of data obtained from the web and transformation ofthat data for storage in a database. The exemplary process includesextracting data from network 20 into a data structure 2810 in which datais arranged and organized. For example, data related to a traffic reportmay be extracted from the Internet to include information on adescription, a main road, a crossroad, a time, a date, and a severityrating. Data structure 2810 is created and organized by use of rules2815 which include text patterns and descriptions which permit thearrangement and organization of data into data structure 2810. Datastructure 2810 is stored in a data file on a database. Data in datastructure 2810 undergoes a transformation in which a first termsubstitution form is applied to create a data structure 2820. Rules 2825are applied during the term substitution to create data structure 2820,including lexical entry of the transformation table. In the trafficreport example, “Rd.” is transformed to be “road”, “I.” is transformedto be “interstate”, and “Rt.” is transformed to be “route”.

Data contained in data structure 2820 is then put in a parsed form in adata structure 2830 according to rules 2835 which apply attribute phrasegrammars for transferred data. In the traffic report example, a“direction”, such as, north, west, south, and east is identified and a“highway identifier” is determined, such as “interstate” or “highway”.Data in data structure 2830 is then placed in a re-arranged form in datastructure 2840 by applying term arrangement rules 2845. Data in datastructure 2840 is manipulated by a second term substitution form andplaced in data structure 2850 by applying rules 2855 from the lexicaltransformation table. For example, the term “St.” is determined to beeither “street” or “saint” based on its locational identifier <streetst.> or <city st.> in the lexical transformation table.

After the lexical transformations are performed, data is placed in datastructure 2860, an unfused, normalized, and tagged format. Datastructure 2860 preferably resides in database 2850. Normalized andtagged format refers to a format including a uniform organization suchthat data can be easily searched and compared and HTML tags. Often, HTMLtags provide information on the type of data, its location, and itslength. Unfused means that the data has not gone through the fusionprocess described with reference to FIGS. 25 and 26.

Advantageously, the data isolation process described with reference toFIG. 28 takes data from the Web and transforms it into a normalized andtagged format in a database. Normalized and tagged data is prepared fororganization, manipulation, and fusion. Advantageously, the dataisolation process is uniform and works for data from a wide range ofdata sources. Thus, generally the process includes obtaining data fromInternet sources, creating a first data file with the obtained data in afirst format, and generating phrases from the obtained data where thephrases are in a second format associated with a specific interface. Awide range of applications may be used to convert the obtained data intothe first and second formats. For example, text patterns, lexicaltransform tables, attribute phrase grammars, and term arrangement rulesmay be used to convert obtained data into a uniform and searchableformat, which is saved in a data file in a database, and then convertthe saved data to an interface specific format. In alternativeembodiments, other patterns, tables, rules, and data manipulationapplications may be used.

FIG. 29 is a functional diagram 2900 illustrating the transformation ofdata from database 170 to a user of voice portal 10 via some userinterface platform (e.g., WAP, Web, phone, ASR, TTF). Data contained indata structure 2860 (shown also in FIG. 29) is put in a parsed form in adata structure 2910 by applying rules 2915 with attribute phrasegrammars for normalized and tagged data. Attribute phase grammars takenormalized and tagged data to create sensible phrases which include theattributes identified. Data from data structure 2910 is then placed indata structure 2920 by applying a term substituted form using rules 2920containing lexical entry transformation tables. In the exemplaryembodiment, the lexical entry transformation tables of rules 2920 listthe data output structure corresponding to a particular interface. Forexample, the term “route” is transformed into “Rt.” for WAP applicationsand transformed into “Route” for telephone applications using speech.Similarly, the term “U.S.” is transformed into “U.S.” for WAPapplications and to “you ess” for phone applications using speech.

Data from data structure 2920 is placed in a re-arranged form in a datastructure 2930 by applying rules 2935 in which term replacement rulesare applied, depending on the output device used. Term rearrangementrules move terms around to the arrangement which best suits differentuser interfaces. Data in data structure 2930 is then placed in a datastructure 2940 in which sentences are generated by applying rules 2945which include phase generation grammars. For example, a sentence may begenerated which says “we have a <severity> traffic incident between<cross location> and <cross location> on <main road>.” Once data is inthe format of data structure 2940, it is prepared for a variety ofoutput interfaces, such as, WAP, Web, phone, and ASR.

Advantageously, the data transformation process described with referenceto FIG. 29 is a uniform process which takes data and prepares it for awide range of user interfaces. For example, the process allows for datato be extracted from web sources and be semantically identified andprepared for speech transmission via a speech interface. At the sametime, the process allows for the same data to be prepared fortransmission to WAP devices or web applications.

FIGS. 30-33 illustrate several operational paths that demonstrateexemplary interactions between the user and voice portal 10. Userinterface 110 preferably makes use of explicit prompting to guide theuser into making appropriate utterances as described with reference toFIGS. 32-33.

FIG. 30 is a flow diagram 3000 depicting an exemplary system overview,including process blocks representing various functionalities of voiceportal 10. In an exemplary execution path, at a block 3010, voice portal10 greets the user by saying, “Welcome to Quack, brought to you byAmerican Express.” Preferably, voice portal 10 uses caller-ID as meansof identifying the user. In a preferred embodiment, phone numbers arestored as a customer attribute in database 170. Alternatively, phonenumbers are stored in a customer database. Voice portal 10 continues bystating, “Hello, Steve Woods. Please say your PIN or enter it on thenumeric keypad. If this is not Steve, please say or enter your phonenumber.” The user then responds verbally “5082” to give his or her PIN.Once authentication is made, voice portal 10 goes to a block 3020. Atblock 3020, voice portal 10 indicates “You are at the Quack runway.Please say the name of the category you are interested in from thefollowing list: movies, weather, traffic, stocks, and sports.” The userresponds with a category name or a goodbye. If a category name isprovided, voice portal 10 goes to a block 3030. If a goodbye is given,voice portal 10 provides a graceful exit to voice portal 10. In anexemplary response, the user says “Weather” and voice portal 10 goes toblock 3030. At block 3030, voice portal 10 says, “Welcome to Weather,brought to you by The Weather Channel,” and goes to a block 3040. Atblock 3040, the identify unique existant subsystem is performed.

After block 3040, a block 3050 is performed in which a determination ismade as to whether the existant was found in the identify uniqueexistant subsystem of block 3040. If the existant was not found, controlreturns to block 3030. If the existant was found, a block 3060 isperformed in which the found existant subsystem (described withreference to FIG. 33) is performed.

Referring now to FIG. 31, the identify unique existant subsystemexecuted at block 3040 (FIG. 30) includes a block 3110 in which database170 provides an attribute from an attribute dependency graph for thecurrent vertical domain (e.g., weather, traffic, movies). If there arenot more attributes in the attribute dependancy graph, control passes toa block 3115 where an existant search fail is noted. After block 3115,control passes to block 3030 (FIG. 30). After block 3110 (FIG. 31), ablock 3120 is executed in which an attribute vocabulary is built from anattribute value set provided by database 170. After block 3120 isperformed, a block 3130 is performed in which voice portal 10 usesautomatic speech recognition (ASR) techniques following a method N toacquire the user's response to an attribute value prompt. For example,voice portal 10 may request the user's location by ZIP code, oneexemplary method N. The user may respond by giving his or her ZIP code,such as, “53045”.

At a block 3140, a decision is made as to whether the voice recognitionwas successful. If not successful, block 3130 is performed with ASRtechniques following a fallback method N+1. For example, in the weathervertical domain, a fallback method N+1 may be to ask for the state andcity of the user's location. In preferred embodiments, fallback methodsinclude choosing an attribute from a list, constraining the attributevalue set by partitioning space (e.g., get state, then city name), andspelling the attribute value. If voice recognition is successful, ablock 3150 is performed in which voice portal 10 searches database 170with the acquired attribute. After block 3150 is performed, a flow chart3200 (FIG. 32) is performed.

Referring now to FIG. 32, flow chart 3200 is shown illustrating aportion of the identify unique existant subsystem. After block 3150 isperformed (FIG. 31), a block 3210 is performed to determine the numberof matching existants from the search of database 170. Different actionsare taken depending on the number of matches found in the search of theproduct database. If no matches are found, a block 3220 is performed inwhich a determination is made as to whether to seek a Compound UniqueKey. A Compound Unique Key may exist if there are one or more uniquekeys or identifiers which are not contained within database 170, but maybe used to find the desired item on the Internet.

If one match is found, a block 3230 is performed in which voice portal10 verifies if the match is the correct existant. If the number ofmatches is greater than one but less than the maximum number for a list,a block 3240 is performed in which the user is requested to identify theexistant from the list of matches. If more matches are found than themaximum number of possible entries in a list, a block 3250 is performedin which it is determined whether the attribute is “extensible.” Inother words, a determination is made as to whether more information canbe provided about the attribute. If more information cannot be provided,control returns to block 3110 (FIG. 31) in which another attribute fromthe attribute dependency graph is obtained. If the attribute isextensible, a block 3260 is performed in which the attribute isattempted to be extended. If the attribute can be extended, controlpasses to block 3120 (FIG. 31) in which a vocabulary set is built andASR techniques and methods are used to obtain an attribute value. If theattribute cannot be extended, control passes to block 3110 (FIG. 31) inwhich another attribute from the attribute dependency graph is obtained.If the extension of the attribute results in a list of items, controlpasses to block 3240.

Referring now to the query performed at block 3220, if a determinationis made that there is not a Compound Unique Key to use for a WWW search,control passes to passes to block 3110 (FIG. 31). If the determinationis made that a Compound Unique Key may exist, control passes to a block3270 in which a determination is made as to whether the WWW is searched.If the WWW is not to be searched, control passes to block 3030 (FIG. 30)which is the top level of the current vertical domain. If the WWW is tobe searched, control passes to a block 3280. Referring now to block 3230and block 3240, if the correct existant is found or the correct existantis found from the list, control passes to block 3280. If the correctextistant is not found in block 3230 or block 3240, control passes toblock 3220 for a determination of whether there is a Compound Unique Keywhich can be searched on to find an item. At block 3280, a web lookup isperformed. At this point, the customer may be presented targetedadvertisement of a variety of lengths. Advertising is described infurther detail with reference to FIG. 36. During block 3280, block 3060is performed in which a found existant subsystem is executed.

Referring now to FIG. 33, the found existant subsystem includes a block3310 in which a log is made into the customer database of the founditem. In a preferred embodiment, customer database is included indatabase 170. After block 3310, a block 3320 is performed in whichinformation is prepared for presentation as appropriate to the verticaldomain from information in database 170. After block 3320, a block 3330is performed in which related information and command grammar is built.For example, in the movie vertical domain, if a list of movies showingat a specific theater is played, grammar would include the movie titlesto allow the user to ask for more information about a particular movie.

At a block 3340, information is returned from the user. In a preferredembodiment, possible acceptable commands include commands to hear moredetailed information, to hear information from a specific source, tohear related information (e.g., cheaper, better), and to take actionappropriate to the vertical domain (e.g., increase the bid, changelocation). After block 3340, a block 3350 is performed in which the nextactivity is obtained. If a new vertical domain is desired, controlpasses to block 3020 (FIG. 30). If a new selection from the top of thecurrent vertical domain is desired, control passes to block 3030 (FIG.30). If a new existant is desired, control passes to block 3040 (FIG.30).

Referring again to FIG. 32, after block 3280 is performed, a block 3290is performed in which web lookup results are coordinated by updatingdatabase 170. During coordination of the web results at block 3290, asmart delay handle may be performed at a block 3295 in whichadvertisements or other forms of handling a delay are performed. Smartdelay handle at block 3295 uses information from the customer databaseand the advertising database. In a preferred embodiment, the customerdatabase and the advertising database are subsets of database 170. Inalternative embodiments, the customer database and the advertisingdatabase are physically separate databases.

In operation, the system and method for voice access to Internet-basedinformation described herein advantageously can identify a verticaldomain of interest to the consumer (e.g., movies, shopping), and then“funnel” user responses from the range of all possible things in avertical domain to the one or set of things that the consumer wants.This funneling within a vertical domain involves system-directedquestioning the user about attributes of products or services, accordingto a set of pre-defined “paths” to funnel to a particular item. Pathsare defined in terms of orderings of constraints about products to beascertained and instantiated.

FIG. 34 illustrates a flow diagram 3400 of the funneling process whichallows voice portal 10 to funnel user responses and achieve highlyaccurate rates of voice recognition for user responses. At step 3410, auser calls voice portal 10. After step 3410, a step 3415 is performed inwhich the caller is identified, using the different possible methodsdescribed above. After step 3415, a step 3420 is performed in which theuser selects a vertical domain of interest. A step 3425 is thenperformed in which the attribute funnel characteristic to the chosenvertical domain of interest is started. After step 3425, a step 3430 isperformed in which voice portal 10 determines whether the user haspreferences in this vertical domain of interest. If there arepreferences and the user does not want to over-ride them, control passesto a step 3460 in which the item or service is indicated as found basedon user preferences.

If no preferences are available or the user over-rides his or herpreferences, a step 3435 is performed in which an attribute vocabularyset is built. Vocabulary sets advantageously allow voice portal 10 tohave a limited number of possible responses from which to use in speechrecognition of user response at this point in the vertical domain ofinterest. With a defined vocabulary set, voice portal 10 advantageouslyachieves with high rates of recognition conventional speech recognitiontechniques. For example, it would be easier to recognize the term“Brewers” after the user has selected Major League Baseball (MLB) teamsand a vocabulary set of possible requests for MLB teams has been built.Such a vocabulary set may include a variety of different types of userinputs for the same information. For example, in the MLB team example, avocabulary set may include all the city or state names associated withMLB teams as well as the MLB team mascot. Thus, “Milwaukee” and“Brewers” would both be part of the vocabulary set of MLB teams.

After an appropriate vocabulary set has been built, a step 3440 isperformed in which voice portal 10 queries on the attribute. Forexample, “What Major League Baseball team would you like to hear about?”After step 3440, a step 3445 is performed in which the attribute isidentified. If an attribute is not identified, a step 3447 may beperformed to carry out fallback procedures for the identification of theattribute. At a step 3450, voice portal 10 determines whether it hasreached an “end state,” or a point at which the item or service has beenfound. If an “end state” has not been reached, a step 3455 is performedin which the next attribute is accessed and control returns to step3430. In the baseball example given, an end state has not been reachedwith only the team name. Other “narrower” attributes, such as, recentgame results, player statistics, team standings, or other relatedinformation must be requested. Once step 3460 is performed, a step 3465is performed in which the found item or service is reported to the user.

In an exemplary embodiment, the user selects item in the followingmanner. The user first specifies the domain of interest (e.g.e-commerce, traffic information, weather information, movies, etc.). Theuser then chooses an item (e.g., a book, a toy, a route of interest fortraffic information, a city of interest for weather information, etc.)by specifying attributes of the item. The user is then provided withdetailed information for an identified item, appropriate to the domainof the item (e.g. products, traffic, weather, movies, etc.). Forexample, in the E-commerce domain of interest reviews, vendorinformation including pricing, shipping costs and availability areavailable. In the movies domain of interest, directory, producer, andcast are provided. In the auctions domain of interest, outstanding bidsare made available.

Advantageously, the user may request information by locale (e.g., thenearest vendor for an identified product, the nearest theater for amovie) with multiple ways to identify a locale (e.g., zip, town name,city area “Boston North, West etc.). In an exemplary embodiment, astrategy for location-identification around ZIP codes is used whichinvolves asking for suburb name, falling back to city or even state andthen zooming in again. In an exemplary embodiment, the user is providedupon request with the date and time on which information was lastupdated. Preferably, all data presented to the user is of currencyappropriate to the domain. The user is informed of the source of theinformation (“provided by XXXXX”) for “pure” source information, orinformation from only one source. In a preferred embodiment, a “help” or“instructions” option is available at every selection point.

The user may request item comparisons based on item attributes, asappropriate to the domain. The user may request identification of“better”, “cheaper” and “related” items, as appropriate to the domain.Advantageously, the user may explicitly record items in a number ofuser-defined lists, as appropriate to the domain of interest. The usermay review items from their lists. The user may request phone or emailnotification of information changes (appropriate to the domain) foritems on their lists.

FIG. 35 illustrates a flow diagram 3500 of an exemplary process ofcarrying out a transaction using voice portal 10. At step 3510, a useraccesses (telephones or calls) voice portal 10. After step 3510, a step3515 is performed in which a funneling process is performed to identifyan item or service desired by the user. Such funneling process performsthe operations illustrated in flow diagram 3400 and described withreference to FIG. 34.

After step 3515, a step 3520 is performed in which voice portal 10 asksthe user to specify a transaction desired and relating to the item orservice identified. After step 3520 is performed, a step 3525 isexecuted in which voice portal 10 identifies the appropriate voiceportal rule to execute the specified transaction. After step 3525, astep 3530 is performed in which the rule is executed to carry out thespecified transaction. Transactions can include purchasing an item orservice, making a bid on an auction, or any other type of transactionpossible over the Internet. After step 3530, a step 3535 is performed inwhich voice portal 10 records the result of the transaction. Preferably,the result is recorded in database 170. After step 3535, a step 3540 isperformed in which the transaction is reported to the user.

Different transactions (e.g., bid, watch, buy, track) are appropriate todifferent domains. For example, in the e-commerce domain of interest,the user may order an identified product from a chosen vendor. Further,the user may add an item to a shopping cart for purchase at a latertime. The user may specify, when ordering, a billing credit card andshipping address (from user profile or manually). The user may alsorequest status information for previously ordered products. As anotherexample, in the auction vertical domain of interest, the user mayincrease existing bids or the user may place bids on new auctions.

Advantageously, the process of carrying out a transaction using voiceportal 10 does not require a user to make any manual actions on acomputer. The user can buy items, make bids, or do any other Internettransaction without clicking a mouse, pressing a key on a computerkeyboard, or any other computer-interface manual action (e.g., mouseclick, keyboard entry). Thus, the process described with reference toFIG. 35 can be a “No Click” Internet transaction process. User canutilize the touch pad of a phone and still perform a “No Click” Internettransaction.

FIG. 36A illustrates a flow diagram 3600A of an exemplary process ofadvertising using voice portal 10. Advantageously, advertising subsystem120 includes a method for determining what advertisements to play to aspecific user. Generally, this method includes setting selectionconstraints based on a context, such as, user demographics, locationdemographics, and a current vertical domain of interest. After selectionconstraints are set, the method queries an advertisement database basedon the constraints and retrieves a list of possible advertisements. Thelist of possible advertisements is re-ordered based on a sales criteriafor each advertisement. An advertisement is selected from the re-orderedlist and presented to the user.

Referring to flow diagram 3600A, in a step 3610A, advertising subsystem120 in voice portal 10 sets selection constraints for advertisements tobe presented to a user. In one embodiment, the selection constraints arebased on user-centric information, such as, user demographics, locationdemographics, and the current selected vertical domain of interest (ifany) as well as advertising-centered information, such as, advertisingsales criteria, lack of repetition, and other advertising effectivenessfactors. Such constraints or criteria are used in selecting from avariety of different types of advertisements, such as, an introductorysponsorship advertisement, a vertical sponsorship advertisement, and acommercial advertisement. After step 3610A, a step 3615A is performed inwhich database 170 is queried for a list of possible advertisementsbased on the constraints selected in step 3610A.

After step 3615A, a step 3620A is performed in which the list ofpossible advertisements is re-ordered based on sales criteria factors.In one embodiment, sales criteria are used to determine the following:(1) Is the advertisement delivery rate being achieved for thisadvertisement? (2) Has the target delivery minimum been achieved forthis advertisement? Advantageously, the sales criteria are used to makesure that each Advertisement customer will have their requirements fordelivery satisfied. In one embodiment, a ratio is calculated toprioritize the advertisements on which should be delivered first.

The following provides an example of using a ratio as the determiningfactor of how advertisements are ordered. Advertisement X needs 100,000deliveries in its contract. Voice portal 10 has already delivered 7,000instances of Advertisement X. The start date of the contract is May 10and end date is June 7th. The current date is assumed to be May 15th.So, an exemplary ratio is determined as follows:

Number of days since start of contract=5.

Length of contract=27 days.

Number of days that the Advertisement needs to be played=22.

Percent of ads played=7,000/100,000˜=7%

Percent of days already played=5/27˜=18.5%

As such, an exemplary final ratio is:

(% of days already played−% of ads played)/Number of days remaining incontract

Advantageously, this ratio accounts for advertisements that should beplayed soon (lower denominator→higher ratio), and the discrepancies ofadvertisements that have already been played get pushed back with alower ratio.

After step 3620A in which the list of possible advertisements arere-ordered, a step 3625A is performed in which an advertisement ischosen. In one embodiment, the advertisement is chosen based on thehighest ratio available in the list of possible advertisements. Afterstep 3625A, different actions are taken depending on the type ofadvertisement to be presented. In a step 3630A, if there is noadvertisement available and if the advertisement type is an introductorysponsorship advertisement, an exception is raised in a step 3635A.Otherwise, a step 3640A is performed in which a decision is made as towhether an advertisement is available. If there is an advertisementavailable, a step 3645A is performed in which the advertisement isplayed. If no advertisement is available, a step 3640A is performed inwhich the selection constraints are reset and control returns to step3620A.

As such, there are differences in the process steps for each type ofAdvertisement, of which there are three: introductory sponsorshipAdvertisements, vertical sponsorship Advertisements, and commercialAdvertisements. The following is an exemplary process for selecting anintroductory sponsorship Advertisement:

-   -   1. Set selection constraints based on location for the        introductory sponsorship Advertisement type (vertical is not        used because, no vertical is applied).    -   2. Query the database based on constraints with result        transformed into a list of possible Advertisements to play.    -   3. Reorder the list based on the sales criteria.    -   4. Choose the Advertisements from the list with the highest        ratio. There must be an Advertisement in the database and raise        an exception otherwise.        The following is an exemplary process for selecting an vertical        sponsorship Ad:    -   1. Set constraints based on user demographics, location        demographics, and vertical type for the vertical sponsorship        Advertisement type.    -   2. Query the database based on constraints with result        transformed into a list of possible Advertisements to play.    -   3. Reorder the list based on the sales criteria.    -   4. Choose the Advertisement from the list with the highest ratio        if one is available and return to the user interface.    -   5. If none are available, reset selection constraints based only        on vertical type and set the type of vertical sponsorship to be        only for Quack promotions.    -   6. Reorder the list based on the sales criteria.    -   7. Choose the Ad from the list with the highest ratio if one is        available and return to the user interface.    -   8. If the user heard all of the Advertisements from the list,        then return the Advertisement that was last played to the user.        If for some reason the list is empty and no Advertisements are        available, raise an exception.        The following is an exemplary process for selecting a commercial        advertisement:    -   1. Set selection constraints based on location demographics,        customer demographics and vertical type for the commercial        Advertisement type.    -   2. Query the database based on those constraints with the result        transformed into a list of possible Advertisements to play.    -   3. Reorder the list based on the sales criteria.    -   4. Choose the Ad from the list with the highest ratio if one is        available and return to the user interface.    -   5. If none are available, reset selection constraints based only        on vertical type and set the type of commercial to be either for        Quack (i.e., voice portal system) commercials or paid        commercials (regardless of type entered).    -   6. Reorder the list based on the sales criteria.    -   7. Choose the Advertisement from the list with the highest ratio        if one is available and return to the user interface.    -   8. If the user heard all of the Advertisements from the list,        then return the Advertisement that was last. If for some reason        the list is empty and no Advertisements are available, then        raise an exception.

Referring now to FIG. 36B, a flow chart 3600B illustrates a secondexemplary process of advertising using voice portal 10. In a step 3610B,a user accesses (telephone or calls) voice portal 10. After step 3610B,a step 3615B is performed in which a user lookup is performed toidentify the user. Caller identification may be done in a variety ofmethods, some of which are described with reference to FIGS. 2 and 30.After step 3615B, in a step 3620B, a determination is made as to whetherthe user is known by voice portal 10. If the user is not known, a step3625B is performed in which a default profile is used for the user. Inan exemplary embodiment, the default profile does not include userconstraints or limitations for certain advertisements. The defaultprofile can be geared for certain parameters known about the call, suchas for example, the area code of the user, time of day of the call, dayof the week, etc. If the user is known or after step 3625B is performed,a step 3630B is performed in which advertising subsystem 120 generates aset “S” of advertisements based on the type of interface (e.g., speech,WAP, WWW) including user constraints particular to the current user.

Given the current operating context (e.g., particular user, verticaldomain of interest), in a step 3635B, advertising subsystem 120generates weights for advertisement set S based on the advertisingcontext. After step 3635B, a step 3640B is performed to determinewhether the context is enough to accurately know what the user mostwants. If the context is not enough, a step 3645B is performed in whichan advertisement is picked based on the partial context obtained. If thecontext is sufficient, a step 3650B is performed in which the best fitadvertisement is played.

Advantageously, advertising subsystem 120 provides an initial generaladvertisement or sponsorship message to all callers. Advertisingsubsystem 120 also provides targeted audio advertisements to users,chosen based on a utility function appropriate to the domain. In anexemplary embodiment, the utility function is related to theavailability of the product or service to be advertised, the relatednessof the current item (e.g., DVD is related to television), the relevanceto the user (e.g., by demographic), the desirability of the user to theadvertiser, and the value to the service provider (e.g., based oncost/return). Advantageously, advertising subsystem 120 is capable ofdelivering a certain number of advertisements to users within a certaintime frame. Moreover, advertising subsystem 120 is capable of deliveringadvertisements across different platforms, such as wireless applicationprotocol (WAP), WWW, and speech interfaces.

Taking for example, the speech interface platform, within the firstminute, one sponsorship advertisement and one targeted advertisement aredelivered to the user. Within each additional 40 seconds, a secondtargeted advertisement is delivered. In one embodiment, a sponsorshipmessage will process in 3-5 seconds, and then a targeted advertisementtakes 10-20 seconds.

The implementation of this structure is based on the fact that theintroductory sponsorship advertisement is presented upon entering thesystem. Each time the user enters a vertical, the user is prompted witha “vertical sponsorship”. Once the user is about to receive the datarequested, a full commercial would be presented to the user.Advantageously, this model approximates the schedule listed inpreviously as the user is estimated to be searching for their piece ofinformation for 40 seconds before receiving it.

In the advertising context, “Speak-throughs” are requests to delivermore detailed information upon presentation of an advertisement.Advantageously, speak-throughs apply not only to speech interface, butalso to both the WAP and WWW. For the WAP, speech and text can beconsidered for a speak-through while clicking on a banner to find outmore on an ad would be a speak-through on the WWW. One embodiment of aspeak-through on a voice interaction is to point the customer to awebsite address or a phone number. In alternative embodiments,speak-throughs collect an email address or a custom phone number toprovide to the advertiser to send more relevant information to thecustomer. With a WWW interface, speak throughs may include using anoutside source to manage and audit customer information. Advertisingsubsystem 120 may also provide targeted “banner” advertisements tousers, chosen based on a utility function appropriate to the domain(e.g., WWW interface).

The management of the advertising delivery of advertising subsystem 120is based on a combination of several factors. In an exemplaryembodiment, an advertisement is delivered in one of three places. First,an advertisement may be delivered when the user is preparing to enterthe system to begin a new session. This sponsorship message will be inthe voice of user interface 110, or “the system voice”, and shouldrotate among several alternative advertising sponsors. For example, asponsor message may say, “Quack is brought to you by Visa, the currencyof the web” or “Quack is brought to you by SprintPCS; the clear futureof cellular services.”

Second, another sponsorship advertisement (a “vertical sponsorship”advertisement) may be delivered just before a user has accessed acertain vertical of the system, such as movies, traffic, or weather. Forexample, such an advertisement may say, “Brought to you by IMDB, theworld's authorities on movie information” and “LCE Sony Metreon:Boston's ONLY choice for great movie selections.”

Third, an advertisement may be delivered just before the user receivesthe refined request. This type of advertisement is defined as a“commercial”. Such advertisements are timely (i.e., delivered at chosenpoints) but only on a so-often basis, such as every 2 minutes.Advantageously, the system voice can point out a value-added situationfor the user that may be helpful. For example, a nearby restaurant maybe suggested when a user is selecting a movie at a particular theater.Speak-through advertisements are preferably used here, althoughnon-speak through advertisements are possible. The advertising contentitself is preferably about 7 seconds in length. The speak-through adsare preferably of maximum possible quality (i.e., professionallyproduced), and of length of about preferably 15 to 20 seconds. Forexample, if the user selects American Beauty as a movie at LCE SonyMetreon, the system voice says, “I'm looking up the listing at SonyMetreon . . . if you would like to hear about ‘Tony's Matriciana’:Boston's best Italian food only 5 minutes from Sony Metreon, say‘Tony's!’, or hold on for your listing.” The user may then automaticallyset up a reservation. Other relatedness attributes may also be used fortargeted advertising. Advantageously, the advertisement is delivered inthese different spots due to the current assumptions that making avertical-specified request will take the amount of time to deliver theadvertisements.

Along with these issues, decisions are made on which advertisementsusers are to have delivered to them. Factors incorporated into thisdecision include the length of the call, what type of vertical contentis requested, combination of content and user profile (and/or location)(i.e. restaurant ads should be local to the customer), revenuepotential, callers requesting specific information, and if the user hasheard the advertisement already. In an exemplary embodiment,advertisements are rotated based on the following factors: When was thelast time that the ad was played? When was the last time the user heardthe ad before this call? Did the user hear the ad already this call? Isthe ad delivery rate being achieved for this ad? Has the target deliveryminimum been achieved for this ad?

Advantageously, advertisements are delivered in a manner such that thepresentation is appropriate to particular customers and are trackedaccordingly to billing rates. Thus, certain basic data is gathered tomanage each ad, such as, how many times has the ad been played and howmany times has an individual user heard the ad?

As well, given this basic data, the following more complex queries areavailable. For example, queries may include the ability to create areport of all users who have heard an ad among various defined groupingsas follows: name, demographic information, location, and relatednessinformation (what else have these users requested). Queries may alsoinclude the ability to create a report of all users who have requestedspeak-through information.

The capability of barging-in (i.e., stopping the advertisement frombeing played), which is possible during other modes of operation forvoice portal 10 can be removed while presenting an ad. The capability toprevent barge-ins is important in that advertisers must be assured ofthe data that has been acquired with respect to the advertising that wasgiven through voice portal 10. In one embodiment, the gathering of thisadvertising data is done by a third party auditor.

Advertising subsystem 120 keeps a record of all advertisements served tousers, including successful (i.e., complete) and unsuccessful (i.e.,incomplete) deliveries. Preferably, this record is stored in database170. Advantageously, advertisements may be targeted by vertical domainof interest, or caller location or user, or user preferences, or userpast interests, or by some other combination of advertiser interests anduser collected information.

Advantageously, the use of context sensitive information can be used tonarrowly target advertisement to a user. Context-sensitive Advertisementtargeting in voice portal 10 relates an Advertisement commercial tonearly exactly what information the user receives. To make this functioncorrectly, an appropriate pointer is passed into the selection algorithmjust before the Advertisement is to be played. In the one embodiment,the vertical type is the context pointer.

In other embodiments, an existant is the context pointer that allowsmore specific targeting. This context pointer matches its attributes'criteria with market research criteria to determine weights in certaincategories. These category weights combined with the sales criteria ofthe Advertisements in the initial list define an ordering of contextweights from which to best select Advertisements. This initial list,created from demographics and vertical type, form the basis for thecontext weighting. Mathematical notation is introduced to generalizethis problem into an algorithm, and an example follows to illustrate it.

First, variables are defined with respect to the parameters involved.Let the list of attributes of the existant passed into the algorithm bedefined by the set {e₁, e₂, e_(m)}, where m is the number of attributesin the existant. For example, for a movie existant, sample attributesare genre, location and show time. The list of categories available toassociate Advertisements with, are defined by the set {C₁, C₂, . . . ,C_(n)}, where n is the total number of categories. Some samplecategories in the system would be: family, restaurants, nightlife,movies, and entertainment. Let there be a context category weight,W_(i), for each category C_(i), where i∈{1, . . . , n}. The purpose ofhaving context category weights is to determine the strength of thecontext in comparison to the advertisement's category weights, asdiscussed below.

The market research criteria for all existants is represented by P={p₁,p₂, p_(t)} where t is the total of all criteria in the database. Eachcriterion p_(j) has an associated weight w_(j), where j∈{1, . . . , t}and each attribute e_(i) will try to satisfy all p_(j), for all i, j,where i∈{1, . . . , m}, j∈{1, . . . , t}. Thus, if e_(i) satisfiesp_(j), and p_(j) belongs to category C_(k), then W_(k)=W_(k)+w_(j),where i∈{1, . . . , m}, j∈{1, . . . , t}, k∈{1, . . . , n}. Thisiteration is used to define the aforementioned context weights of eachcategory.

Once the total context weight for each category, W_(k), is defined, itsassociated strength ratio, R_(k), must be calculated. The strength ratioof a category is used to determine if the context of the existant isstrong enough to merit in the selection of the Advertisement. Forexample, if the family category has many criteria in P, then we want tomake sure that the weights corresponding to the context of the existantare in an acceptable proportion. So, R_(k)=W_(k)/T_(k), where T_(k) isthe total weight of all criteria in P relating to category k.

The list of advertisements generated by the demographic query aredefined by the set A={A₁, A₂, . . . , A_(r)}, where r is the totalnumber of ads in the list. Each ad A_(i) has its own category weightx_(k), where i∈{1, . . . , r} and k∈{1, . . . , n} which is used inconjunction with the algorithm's corresponding context category weightratio R_(k).

Thus, once the initial list of Advertisements A has been created byfiltering the demographics and Advertisement type on the database, thesteps in the algorithm would be as follows:

-   -   1. Set category weights, W_(k), for each category C_(k) as        follows        -   Initialize each W_(k)=0, where k∈{1, . . . , n}.        -   For each i∈{1, . . . , m} and for each j∈{1, . . . , t},            based on the current attributes of the existant, {e₁, e₂, .            . . , e_(m)}, if e_(i) satisfies p_(j), and p_(j) is            associated with category C_(k), then W_(k)=W_(k)+w_(j),            where k∈{1, . . . , n}    -   2. Now tabulate the categories' total weights independent of the        attributes of the existant. From those total weights, establish        each category's context ratio        -   For each k∈{1, . . . , n} and for each j∈{1, . . . , t}, if            p_(j) is associated with category C_(k), then            T_(k)=T_(k)+w_(j). Set the context category context ratio,            R_(k)=W_(k)/T_(k)    -   3. For each category k, multiply each R_(k) by the category        weight x_(k) of each Ad A_(i), and then multiply the sum by the        sales criteria ratio of the Ad, S_(i), to get context total        G_(i)        -   For each i∈{1, . . . , r}, calculate G_(i) where

G _(i) =S _(i)·(R ₁ x ₁ + . . . +R _(n) x _(n))

-   -   4. Select the Ad A_(i), where i is defined by max(G_(i)), i∈{1,        . . . , r}.        The above algorithm can be illustrated by a simple example.        Consider an example where a user is using the service of voice        portal 10 while in the movies vertical domain of interest. The        vertical sponsorship Advertisement has been played and the user        is just about to receive information regarding a movie showing.        Thus as context, the selection includes the pointer to the        specific existant that is to be played, which is for arguments'        sake, “Mission to Mars”. Some of the attributes of the movie        showing existant are rating (e.g., R), genre (e.g., Thriller)        and a show time (e.g., 4:00 pm), which can be represented by        {e₁, e₂, e₃}. Thus, there needs to be a list of matching context        criteria containing the elements P={p₁, p₂, . . . , p_(t)}. A        sample list of criteria could be represented in the database as:

Attributes Match Type Value Category Weight Movie — — Entertainment 10Movie Rating = G Family 80 Movie Rating = R Nightlife 50 Movie Genre =Suspense Teen 40 Movie Genre = Suspense Adult 50 Movie Genre = ThrillerTeen 80 Movie Show Time > 7:00 PM Teen 80 Movie Show Time < 4:00 PMFamily 70From this table, the categories can be inferred as C={Entertainment,Family, Nightlife, Teen, Adult} where k=5. So from step 1, W₁=10, W₂=0,W₃=50, W₄=80 and W₅=0. From step 2, establish R₁=1, R₂=0, R₃=1, R₄=0.4and R₅=0 (this is assuming that P has only 8 elements, which will notlikely be the case, and will be around 200 or more elements). Now,assume that the Advertisement list A has three Advertisements. Assumethat the weightings of the advertisements' five categories are:

Sales Ad Name Entertainment Family Nightlife Teen Adult Ratio IMDB 0.90.5 0.7 0.9 0.9 0.8 Mission 0.9 0 0.9 0.9 0.8 1.1 to Mars AMC 0.9 0.90.7 0.9 0.7 1.0 TheatersSo, from these weightings, {x₁, x₂, x₃, x₄, x₅}, we can make thecalculations to get values for G_(i) for each Ad A_(i), where i∈{1, 2,3}, as follows:

G _(i) =S _(i)·(R ₁ x _(i) + . . . +R _(n) x _(n))

G ₁=0.8·((1)(0.9)+0+(1)(0.7)+(0.4)(0.9)+0)=1.568

G ₂=1.1·((1)(0.9)+0+(1)(0.9)+(0.4)(0.9)+0)=2.376

G ₃=1.0·((1)(0.9)+0+(1)(0.7)+(0.4)(0.9)+0)=1.96

Thus, based on context and sales ratio determination, the “Mission toMars” Advertisement is most appropriate. This algorithm notes thecontext of different categories based on their relevance to the piece ofinformation being retrieved. It also organizes the fact that theAdvertisments that need to be played for sales criteria are noted andfactored into the ordering. The example illustrates only a short list ofads, categories and criteria in P. The algorithm is intended to takeadvantage of many more categories and criteria.

FIGS. 37-43 illustrate an exemplary dialog map of the interactionbetween the user and voice portal 10. The dialog map described withreference to FIGS. 37-43 is for illustration purposes only. While onlymovies, weather, traffic, stocks, and sports vertical domains ofinterest are shown in the FIGURES, it should be apparent that anyvertical domain of interest could be included in such a dialog map (andin the interaction of voice portal 10 with the user), particularly inlight of the expansive and adaptive capabilities available due to datastructure models 300, 400, and 450, described with reference to FIGS.4-6. Furthermore, the specific blocks representing differentinteractions between the user and voice portal 10 are for illustrationonly. A wide variety of interactions are possible for each of the manypossible vertical domains of interest.

FIG. 37 illustrates a dialog map 3700 in which after a telephone call ismade by a user to voice portal 10, a block 3710 is performed in which awelcome is provided. After block 3710, a block 3720 is performed inwhich a sign-in procedure is followed (described further with referenceto FIG. 38). After the sign-in procedure of block 3720, the user canselect to have an introduction to the service of voice portal 10 atblocks 3730 and 3740 or go directly to the runway information forintroduction of the possible vertical domains of interest at a block3750. Specifically, at block 3730, introductory information is providedon the service provider. At block 3740, introductory information isprovided on how the service works. At block 3750, voice portal 10requests the user to select a domain of interest from a “runway” (e.g.,movies, weather, traffic, stocks, sports).

If the user selects the movies domain of interest, a block 3760 isperformed in which a movies subsystem is executed (described furtherwith reference to FIG. 39) and the user has access to movie informationand transactions, such as, movie listings, theaters, and reviews. If theuser selects the weather domain of interest, a block 3770 is performedin which a weather subsystem is executed (described further withreference to FIG. 40) and the user has access to weather information,such as, today's forecast or the extended forecast for a preferredlocation or any location. If the user selects the traffic domain ofinterest, a block 3780 is performed in which a traffic subsystem isexecuted (described further with reference to FIG. 41) and the user hasaccess to traffic information, such as, reports by city, reports for acertain route, or personalized reports. If the user selects the stocksdomain of interest, a block 3790 is performed in which a stockssubsystem is executed (described further with reference to FIG. 42) andthe user has access to stocks information and transactions, such as,market summaries, stock quotes, stock news, and personalized stock newsor transactions (e.g., buy, sell). If the user selects the sports domainof interest, a block 2500 is performed in which a sports subsystem isexecuted (described further with reference to FIG. 43) and the user hasaccess to sports information and transactions, such as, sports scores,sports news, sports events ticket information, and sports fantasy leaguetransactions.

Referring now to FIG. 38, a sign-in subsystem is shown. At a block 3810,caller identification is attempted. One type of user of voice portal 10is an unidentified user. The unidentified user calls in (possibly forthe first time) and is either locatable by conventionalcaller-identification techniques (“caller ID”) or not. If the caller-IDdoes not exist in database 170, the caller is possibly a new caller. Ifthe caller-ID is suppressed, voice portal 10 can't tell either way. Inone embodiment, voice portal 10 asks for a phone number (or otheridentifier) and continues with an “identified” caller. In an alternativeembodiment, voice portal 10 continues without verification. Thisdecision may depend on the kinds of information being requested. Forexample, in particular vertical domains, the identity of the user mayneed to be determined to identify the user before proceeding (e.g.,auctions).

Identified users are either subscribed or non-subscribed. If theidentified user is subscribed, voice portal 10 has information on theuser, such as, credit cards and preferences from database 170.Preferably, a user subscribes so that voice portal 10 can start trackingpreferences and interests to achieve a higher degree ofconsumer-value-added and thus loyalty to the service. The user mayspecify profile information, including addresses and credit cardnumbers, upon subscription. Further, the more information amassed abouta particular caller, the more directed (and hence valuable)advertisements are.

If caller identification is possible, a block 3820 is performed in whichuser confirmation is performed by asking the user for a password. Oncethe password is verified, user preferences can be set and control passesto a block 3870 and control returns to FIG. 37 where an introduction orrunway selection is performed. If the password given is invalid, controlpasses to a block 3840.

If caller identification is not possible or the user did not know his orher password, control passes to a block 3830 where voice portal 10determines the user's account status. If the user does not have anaccount, control passes to a block 3850 where an account setup reminderis given that the user should set up an account. If the user has anaccount, control passes to block 3840 where voice portal 10 gets theuser's account number. If the user has forgotten the account number,control passes to block 3850 where the user is asked to set up anaccount. If the user provides a valid account number, control passes toblock 3820 for user confirmation. If the user gives an account numberwhich is invalid, control passes to block 3860 where voice portal 10informs the user that the account is invalid and to visit a web site orcall a support number for assistance. Control then passes to block 3880and to FIG. 37 where an introduction or runway selection is performed.

Referring now to FIG. 39, a movies subsystem is performed. At a block3910, voice portal 10 plays an introduction to the movies domain ofinterest. The user can select options, such as, movies at a theater,listings for a movie, and movie reviews. If the user selects movies at atheater, control passes to a block 3915 in which voice portal 10determines the geographic location desired by the user. Various methodsmay be employed to determine location, such as, ZIP code, state andcity, or preferences. If there is no theater near the given location, ablock 3920 is performed in which a message is played to inform the userthat no theater is located in the given area. After location isdetermined, a block 3925 is performed in which theater names within thelocation are listed. After block 3925, a block 3930 is performed inwhich movies are listed which are playing at theaters within the area.Voice portal 10 requests the user select a movie and control passes to ablock 3935.

Returning now to block 3910, which played an introduction to the moviesdomain of interest, if the user requests listings for a movie, a block3940 is performed in which voice portal 10 requests the movie title fromthe user. After block 3940, a block 3945 is performed in which thegeographic location desired by the user. As discussed above, a varietyof methods may be used to determine the caller's location. If there aretheaters showing the selected movie, a block 3950 is performed in whichthe theaters playing the movie are listed and the user is asked toselect from the list. Control then passes to block 3935. If there are notheaters showing the selected movie, a block 3955 is performed and thetimes for the movie at the closest locations are provided to the user.Control then passes to block 3935.

Referring now to block 3910, which played an introduction to the moviesdomain of interest, if the user requests a movie review, a block 3960 isperformed in which voice portal 10 requests the movie title from theuser. After block 3960, a block 3965 is performed in which the review isplayed for the selected movie. After block 3965, a block 3970 isperformed in which voice portal 10 asks the user if he or she would liketo find showings for the movie selected. If the user declines, controlreturns to block 3960 to get another movie title for a movie review. Ifthe user accepts, control passes to block 3945.

At block 3935, voice portal 10 provides movie showing times for theselected movie and theater. At a block 3980, voice portal 10 requests anext action. The user can request the theater address, which is thenprovided at a block 3985. The user can also request a review of themovie, which is then provided at a block 3990. Once the user wants toleave the movie domain of interest, control returns to block 3750 ofFIG. 37.

Referring now to FIG. 40, a weather subsystem is performed, as shown. Ata block 4010, voice portal 10 plays an introduction to the weatherdomain of interest. After the introduction is played at block 4010,control passes to a block 4020 in which voice portal 10 obtains locationinformation to use in the weather domain of interest. As describedabove, multiple methods are possible to obtain location information,such as, obtaining location from ZIP code, city or state, and otherlocational indicia. After block 4020, control passes to a block 4030 inwhich voice portal 10 provides a prompt as to whether the user wouldlike live weather information or weather information for later periodsof time. If the user selects to hear weather information for laterperiods of time, control passes to a block 4040 in which voice portal 10plays a prompt which provides weather latency options to the user. Ifthe user wants current weather information or after the user hasselected latency options at a block 4040, control passes to a block 4050in which voice portal 10 provides the weather information desired.

After block 4050 is performed, control passes to a block 4060 in whichvoice portal 10 asks the user whether an extended forecast is desired.If a extended forecast is desired, control passes to a block 4070 inwhich voice portal 10 provides an extended forecast. After block 4070 orif the user does not want an extended forecast, control passes to block4080 in which voice portal 10 asks for the next action from the user. Ifthe user wants to continue in the weather domain of interest, controlpasses to block 4020. If the user wants to leave the weather domain ofinterest, control passes to a block 4090 corresponding to the runwaydescribed with reference to FIG. 37 as block 3750.

Referring now to FIG. 41, a traffic subsystem is performed. At a block4110, voice portal 10 plays an introduction to the traffic domain ofinterest. After block 4110, control passes to a block 4115 in whichvoice portal 10 obtains location information for user or personalizedinformation on the user. After block 4115, control passes to a block4120 in which voice portal 10 obtains city traffic information. If nocity traffic information is possible, control passes to a block 4135 inwhich ZIP code traffic information is obtained. ZIP code trafficinformation is a fallback if a city is not recognized by voice portal10. If the city data is not found and data is contained for locationsnearby, control passes to a block 4140 in which voice portal 10 asks fora nearby city. If at block 4120 no traffic events are reportable,control passes to a block 4125 in which the user is told no traffic isreportable in the city. If at block 4120 no traffic data is available,control passes to a block 4130 in which the user is provided the optionto try another city or to go to the runway for selection of a new domainof interest.

After block 4120, control passes to a block 4145 in which voice portal10 requests a specific traffic route or the “whole city.” After block4145, control passes to a block 4150 in which voice portal 10 obtainsroute direction information. After block 4150, control passes to a block4155 if there is no traffic on the route to report. At block 4155, theuser is given the option to select a new traffic route or the “wholecity” as well as going to the runway and selecting a new domain ofinterest. If route traffic information is available, after block 4150,control passes to a block 4160 in which voice portal 10 lists the routetraffic for the selected route. If the user has selected “whole city” inblock 4145, control passes a block 4165 in which voice portal 10 listscity traffic information.

After block 4160 and block 4165, control passes to a block 4170 in whichvoice portal 10 provides the desired traffic report to the user. Afterblock 4170, control passes to a block 4175 in which voice portal 10 asksfor the next action to be performed in the traffic domain of interest.In an exemplary embodiment, next actions may include repeat the trafficreport, continue the list of traffic information, and going to therunway. After the user has made the selection at block 4175, controlpasses to an appropriate block. For example, if the user selects torepeat the traffic report, control passes to a block 4170. If the userselects the continue list option, control passes to either block 4160 orblock 4165, depending on the selection of a specific traffic route orthe “whole city” made at block 4145. If the user selects go to runway,the control passes to a block 4180 which corresponds to the runwaydescribed with reference to FIG. 37 as block 3750.

Referring now to FIG. 42, a stocks subsystem is performed. At a block4210, voice portal 10 plays an introduction to the stocks domain ofinterest. After block 4210, control passes to a block 4215 in whichvoice portal 10 provides the user with the option of selecting for amarket summary, stock quotes, or a personalized listing called“MyQuack.” If the user selects market summary, control passes to a block4240 in which a market summary is provided for a variety of markets,such as, the Dow Jones Industrial Average, NASDAQ, S&P 500, NYSE Volume,NASDAQ Volume, and 30 year bonds. If the user selects stock quotes,control passes to a block 4220 in which voice portal 10 obtains aspecific stock name from the user. After block 4220, control passes to ablock 4225 in which voice portal 10 obtains the stock exchangecorresponding to the stock name provided in block 4220. After theexchange is identified, control passes to a block 4230 in which voiceportal 10 provides stock information, such as, value, last trade,change, volume, and day high/low.

After block 4230, control passes to a block 4235 in which voice portal10 asks for the next action to be performed in the stocks domain ofinterest. In an exemplary embodiment, the user may select to repeatstock information/continue listing stock information, get a new stock,hear market summary, or go to runway. Depending on the selection made bythe user at block 4235, control passes to block 4240 for market summary,block 4220 for a new stock name, a block 4250 for a personalized myquack stock, or a block 4275 for the runway. Before block 4275, controlmay pass to a block 4270 in which voice portal 10 provides a preferencereminder to the user that preferences may be set to obtain personalizedinformation in a quicker fashion. If the user has already been remindedin this call about preferences, control passes directly to block 4275.

If in block 4215 the user selects “MyQuack” control passes to a block4245 if no account information is identified and block 4250 if accountinformation is identified. In block 4245 a preference setup and accountinformation is established. A suggestion may be made to the user that anaccount be setup on the Web. At block 4250, personalized stockinformation is provided, such as, value, last trade, change, and volume.During operation of block 4250, the user may identify a specific stockby saying, for example, “that one” during the playing of information onthe particular stock. If such a selection is made, control passes to ablock 4255 in which stock news options are listed for the particularstock. After the user has selected a particular type of stock news fromthe list in block 4255, control passes to a block 4260 in which voiceportal 10 plays the selected stock news. After block 4260, controlpasses to block 4265 in which voice portal 10 asks the user whether toreturn to get list of stock news (block 4255) or exist stock news. Ifthe user selects to exist the stock news, control passes to block 4235in which the next action for stocks is requested. Once the user hascompleted the stocks domain of interest, control passes to block 4275corresponding to the runway described with reference to FIG. 37 as block3750.

Referring now to FIG. 43, a sports subsystem is performed. At a block4310, voice portal 10 plays an introduction to the sports domain ofinterest. After block 4310, control passes to a block 4315 in whichvoice portal 10 obtains the type of sports desired by the user or theuser may say “MyQuack” to get scores for a personalized sports type. Ifthe user chooses a particular sport, control passes to a block 4320 inwhich voice portal 10 obtains the league name of a selected sport from alist. For example, voice portal 10 may list “NFL, NBA, NHL, and MajorLeague Baseball.” After the user has made a selection of the leaguename, control passes to a block 4325 in which voice portal 10 obtainsthe specific team the user is interested in. After block 4325, controlpasses to a block 4330 in which sports scores are provided. For example,voice portal 10 may say, “The last game played by TEAM was DATE with afinal score of TEAM 1, SCORE 1, TEAM 2, SCORE 2.”

If in block 4315 the user has selected “MyQuack,” control passes toblock 4340. At block 4340, voice portal 10 provides sports scores forthe personalized MyQuack sports teams. After block 4340, control passesto a block 4335 in which voice portal 10 provides sports news for teamsspecific news. After block 4330 and block 4335 control passes to a block4345 in which voice portal 10 asks whether the user would like thesports information just heard to be repeated. If the user respondsaffirmatively, voice portal 10 returns to repeat the informationprovided. If the user does not want the information repeated, controlpasses to a block 4350 in which voice portal 10 asks for the next actionto be performed in the sports domain of interest. After block 4350,control passes to block 4320 to select a league name, block 4340 toprovide my quack sports scores, or to a block 4355 for runwayinformation. A block 4355 corresponds to the runway described furtherwith reference to FIG. 37 as block 3750. Each subsystem of FIGS. 40-43is shown by way of example only.

While the embodiments illustrated in the FIGURES and described above arepresently preferred, it should be understood that these embodiments areoffered by way of example only. Other embodiments may include variousdata structures for simplifying access to the Internet via voiceportals. The invention is not limited to a particular embodiment, butextends to various modifications, combinations, and permutations thatnevertheless fall within the scope and spirit of the appended claims.

1. A method for executing a transaction related to an item or a serviceusing a telephone, the method comprising: configuring ahardware-implemented voice portal to provide a telephonic user interfaceto a telephone user; storing user demographic information in one or moredatabase; receiving instructions from a user in the voice portal, saidinstruction providing information identifying the item or the serviceand providing a query as to a transaction to be performed, thetransaction being related to the identified item or service; configuringsaid hardware-implemented voice portal to provide to a server system arequest to execute the transaction related to the identified item orservice in response to a user answer; configuring saidhardware-implemented voice portal with an advertising submoduleconfigured to selectively provide the user interface with advertisementstargeted to particular users based on information about the user,wherein the advertising submodule is configured for settingadvertisement selection criteria based on user demographics; whereby thetransaction is executed without the user performing a single action on acomputer interface.
 2. The method of claim 1, wherein identifying theitem or service comprises selectively identifying a category of itemsand services.
 3. The method of claim 2, wherein the step of selectivelyidentifying a category of items and services comprises selectivelyidentifying an attribute of the category of items and services.
 4. Themethod of claim 1, wherein the transaction performed is a purchase. 5.The method of claim 1, wherein the transaction performed is a bid. 6.The method of claim 1, wherein the step of selectively providing anadvertisement comprises providing an advertisement on an item or servicewhich is different from the identified item or service.
 7. The method ofclaim 6, further comprising providing a query associated with theadvertised item or service as to a transaction to be performed.
 8. Themethod of claim 6, wherein the advertising subsystem is furtherconfigured for: querying said database for a list of possibleadvertisements based on said selection criteria, resulting in potentialadvertisements; ordering potential advertisements based on: whether saidpotential advertisements have been delivered a target minimum number oftimes; and whether a desired delivery rate has been achieved for saidpotential advertisements; delivering advertisements via said userinterface based on said ordering.
 9. The method of claim 1, furthercomprising configuring the advertising subsystem for querying saiddatabase to determine if said database contains pre-existing criteriainformation for said telephone user; configuring the advertisingsubsystem for generating a first set of advertisements; configuring theadvertising subsystem for generating weights for said first set ofadvertisements based a context of said advertisements; configuring theadvertising subsystem for determining whether said context is enough toaccurately know what advertisement from the first set of advertisementsthe user most wants; configuring the advertising subsystem for decidingif said context is enough to accurately know what the user most wants;configuring the advertising subsystem for playing said advertisementfrom the first set of advertisements if said context is enough toaccurately know what the user most wants.
 10. The method of claim 1,further comprising creating a record of advertisements that were playedfrom said first set of advertisements.
 11. The method of claim 1,further comprising recording whether or not an advertisement that wasplayed from said first set of advertisements was completed.
 12. Themethod of claim 1, further comprising recording whether or not anadvertisement that was played from said first set of advertisementssubject to a user selecting to hear more about a subject of saidadvertisement.
 13. The method of claim 1, further comprising: coupling anetwork interface to the at least one database; and configuring saidnetwork interface being configured for: accessing the item and serviceinformation over the Internet; processing requests related to the itemand, service information; and carrying out transactions involving theidentified item or service.
 14. The method of claim 13, furthercomprising configuring a fusion engine to compare information obtainedfrom at least one web site and selectively establish canonical datafiles corresponding to information gathered from multiple web sites. 15.The method of claim 13, further comprising: configuring a customermanager module to record user information associated with userpreferences and user behavior related to the service; and configuringsaid customer manager module for providing user information to the userinterface such that the user interface may personalize the service forparticular users.
 16. The method of claim 13, further comprising:configuring an existant subsystem coupled to the at least one database,the existant subsystem being configured to manage all information intoand out of the at least one database.
 17. The method of claim 13,further comprising configuring said user for: updating said user'spreferences during said voice communications; and accepting the input ofsaid user's credit card information and storing said credit cardinformation.
 18. The method of claim 13, further comprising configuringsaid advertising subsystem for creating a record of advertisements thatwere played from said first set of advertisements.
 19. The method ofclaim 13, further comprising configuring said advertising subsystem forrecording whether or not an advertisement that was played from saidfirst set of advertisements was completed.
 20. The method of claim 13,further comprising configuring said advertising subsystem for recordingwhether or not an advertisement that was played from said first set ofadvertisements subject to a user selecting to hear more about a subjectof said advertisement.
 21. The method of claim 13, further comprisingconfiguring said advertising subsystem for asking a series of questionsthat allow the user to uniquely identify the product they are interestedin.
 22. The method of claim 13, further comprising configuring saidadvertising subsystem for returning product pricing and availabilityinformation from various online retailers.
 23. The method of claim 13,further comprising configuring said advertising subsystem for orderingproducts and services.
 24. The method of claim 13, further comprisingconfiguring said advertising subsystem for comparing similar productsand services on different web sites.
 25. The method of claim 1, whereinsaid telephonic user interface comprises a word-based automatic speechrecognition (ASR) engine for speech-based inputs, and wherein saidtelephonic user interface is further configured for accepting text-basedinputs via a keypad if said speech-based input fails.
 26. The method ofclaim 15, further comprising configuring said customer manager formaintaining user preferences based on the items and services involved insaid transactions and the pricing information associated with saidtransactions.
 27. The method of claim 15, further comprising configuringsaid customer manager for maintaining user demographic information byanalyzing a history of said transactions.
 28. The method of claim 15,further comprising configuring said customer manager for pulling userpreferences from users' personalized web pages.
 29. The method of claim15, further comprising configuring said customer manager for creatinguser preferences by programming a spider to search a network for newpreferences likely to be associated with existing user preferences. 30.The method of claim 15, further comprising configuring said customermanager for creating user preferences by programming a spider to searcha network for new preferences likely to be associated with existing userpreferences.
 31. The method of claim 15, further comprising configuringsaid customer manager for storing a personal identification number forsaid users and authenticating said user using said personalidentification number before completing a transaction.